[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:48 dist-test-slave-sb2z krb5kdc[30336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:48 dist-test-slave-sb2z krb5kdc[30336](info): set up 2 sockets
Aug 07 18:20:48 dist-test-slave-sb2z krb5kdc[30336](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:20:50.135761 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.439s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:20:50 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:20:50.364949 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:42559
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:42559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:50.712731 30360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:50.713073 30360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:50.713258 30360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:50.721030 30360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:50.721159 30360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:50.721251 30360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:50.721381 30360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:50.721472 30360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:50.732012 30360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:42559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:42559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:50.733598 30360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:50.735069 30360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:50.748704 30368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:50.748726 30374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:50.748700 30369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:50.749567 30360 server_base.cc:1034] running on GCE node
I20260807 18:20:50.750454 30360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:50.752262 30360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:50.753549 30360 hybrid_clock.cc:648] HybridClock initialized: now 1786126850753501 us; error 94 us; skew 500 ppm
I20260807 18:20:50.754035 30360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:50 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:50.757995 30360 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:20:50.759835 30360 webserver.cc:460] Webserver started at http://127.29.156.254:40417/ using document root <none> and password file <none>
I20260807 18:20:50.760608 30360 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:50.760776 30360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:50.761152 30360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:50.764526 30360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "04ff37b386d74f2b83ba396a8f07aea0"
format_stamp: "Formatted at 2026-08-07 18:20:50 on dist-test-slave-sb2z"
server_key: "f56a3ecf77c13848743e66bedfe4d59f"
server_key_iv: "f4ec246554e520ccde8c5e22b719be4b"
server_key_version: "encryptionkey@0"
I20260807 18:20:50.765352 30360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "04ff37b386d74f2b83ba396a8f07aea0"
format_stamp: "Formatted at 2026-08-07 18:20:50 on dist-test-slave-sb2z"
server_key: "f56a3ecf77c13848743e66bedfe4d59f"
server_key_iv: "f4ec246554e520ccde8c5e22b719be4b"
server_key_version: "encryptionkey@0"
I20260807 18:20:50.771013 30360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 18:20:50.775056 30385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:50.776865 30360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:20:50.777136 30360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "04ff37b386d74f2b83ba396a8f07aea0"
format_stamp: "Formatted at 2026-08-07 18:20:50 on dist-test-slave-sb2z"
server_key: "f56a3ecf77c13848743e66bedfe4d59f"
server_key_iv: "f4ec246554e520ccde8c5e22b719be4b"
server_key_version: "encryptionkey@0"
I20260807 18:20:50.777398 30360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:50.794548 30360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:50.800693 30360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:50.801129 30360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:50.824347 30360 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:42559
I20260807 18:20:50.824368 30461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:42559 every 8 connection(s)
I20260807 18:20:50.826145 30360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:50.827939 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30360
I20260807 18:20:50.828284 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:20:50.830278 30323 external_mini_cluster.cc:1428] Setting key df4014e55deb12625e144c94f5ceffb5
I20260807 18:20:50.831117 30462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:50.844960 30462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0: Bootstrap starting.
I20260807 18:20:50.848959 30462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:50.850193 30462 log.cc:824] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:50.854003 30462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0: No bootstrap required, opened a new log
Aug 07 18:20:50 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:50.863992 30462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } }
I20260807 18:20:50.864619 30462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:50.864769 30462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ff37b386d74f2b83ba396a8f07aea0, State: Initialized, Role: FOLLOWER
I20260807 18:20:50.865700 30462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } }
I20260807 18:20:50.866076 30462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:50.866254 30462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:50.866470 30462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:50.869992 30462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } }
I20260807 18:20:50.870764 30462 leader_election.cc:302] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04ff37b386d74f2b83ba396a8f07aea0; no voters:
I20260807 18:20:50.871678 30462 leader_election.cc:288] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:50.871810 30471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:50.875006 30471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [term 1 LEADER]: Becoming Leader. State: Replica: 04ff37b386d74f2b83ba396a8f07aea0, State: Running, Role: LEADER
I20260807 18:20:50.875567 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:50.834811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.29.156.254:42559)
0807 18:20:50.835830 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:50.835849 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:50.837129 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 18:20:50.846615 (+ 9486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:50.846644 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:50.847093 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:50.847551 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:50.850990 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:50.853074 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:50.855014 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:50.856194 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:50.856436 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:50.862698 (+ 6262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:50.862711 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:50.862734 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:50.862787 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:50.867795 (+ 5008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:50.868827 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:50.868838 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:50.868848 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:50.869137 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:50.869728 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:50.869737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:50.869744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:50.870695 (+ 951us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:50.871184 (+ 489us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:50.872680 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0807 18:20:50.873414 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":453,"threads_started":1}
I20260807 18:20:50.875698 30471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } }
I20260807 18:20:50.878697 30462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:50.879154 30473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04ff37b386d74f2b83ba396a8f07aea0. Latest consensus state: current_term: 1 leader_uuid: "04ff37b386d74f2b83ba396a8f07aea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } } }
I20260807 18:20:50.879592 30473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:50.880225 30474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04ff37b386d74f2b83ba396a8f07aea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04ff37b386d74f2b83ba396a8f07aea0" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42559 } } }
I20260807 18:20:50.880602 30474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:50.881162 30477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:50.889667 30477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:50.902400 30477 catalog_manager.cc:1347] Generated new cluster ID: 9671ac5d08ef46c4bcbd60e54fcebb97
I20260807 18:20:50.902639 30477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:50.923352 30477 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:50.924896 30477 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:50.937574 30477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04ff37b386d74f2b83ba396a8f07aea0: Generated new TSK 0
I20260807 18:20:50.938544 30477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:20:51.000000 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:51.378737 30503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:51.379146 30503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:51.379263 30503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:51.388190 30503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:51.388319 30503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:51.388437 30503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:20:51.398838 30503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:51.400218 30503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:51.401549 30503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:51.415212 30510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:51.415462 30511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:51.415632 30513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:51.416802 30503 server_base.cc:1034] running on GCE node
I20260807 18:20:51.417502 30503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:51.418596 30503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:51.419816 30503 hybrid_clock.cc:648] HybridClock initialized: now 1786126851419770 us; error 32 us; skew 500 ppm
I20260807 18:20:51.420158 30503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:51 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:51.423777 30503 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:20:51.425493 30503 webserver.cc:460] Webserver started at http://127.29.156.193:46153/ using document root <none> and password file <none>
I20260807 18:20:51.426224 30503 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:51.426389 30503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:51.426745 30503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:51.430058 30503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "94f5e7f4b8074f4d86a157881b57d2a8"
format_stamp: "Formatted at 2026-08-07 18:20:51 on dist-test-slave-sb2z"
server_key: "4b2203b6756d59e3aef6b6bdd8d4e29e"
server_key_iv: "d7029e53e2ca0980bb512aa2ca492778"
server_key_version: "encryptionkey@0"
I20260807 18:20:51.430881 30503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "94f5e7f4b8074f4d86a157881b57d2a8"
format_stamp: "Formatted at 2026-08-07 18:20:51 on dist-test-slave-sb2z"
server_key: "4b2203b6756d59e3aef6b6bdd8d4e29e"
server_key_iv: "d7029e53e2ca0980bb512aa2ca492778"
server_key_version: "encryptionkey@0"
I20260807 18:20:51.436483 30503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:20:51.440519 30521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:51.441974 30503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:20:51.442189 30503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "94f5e7f4b8074f4d86a157881b57d2a8"
format_stamp: "Formatted at 2026-08-07 18:20:51 on dist-test-slave-sb2z"
server_key: "4b2203b6756d59e3aef6b6bdd8d4e29e"
server_key_iv: "d7029e53e2ca0980bb512aa2ca492778"
server_key_version: "encryptionkey@0"
I20260807 18:20:51.442433 30503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:51.458746 30503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:51.463282 30503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:51.463696 30503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:51.464848 30503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:51.466368 30503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:51.466495 30503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:51.466681 30503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:51.466768 30503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:51.510331 30503 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:42855
I20260807 18:20:51.510380 30682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:42855 every 8 connection(s)
I20260807 18:20:51.511801 30503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:20:51.514482 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30503
I20260807 18:20:51.514761 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:20:51.515612 30323 external_mini_cluster.cc:1428] Setting key 6108299c5f4773c984dc9c97f2fec8b4
Aug 07 18:20:51 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:51.535626 30685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:51.514744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:60659 (local address 127.29.156.254:42559)
0807 18:20:51.515323 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:51.515329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:51.516898 (+ 1569us) server_negotiation.cc:407] Connection header received
0807 18:20:51.519459 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:51.519465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:51.519588 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:51.519732 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:51.521589 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:51.522384 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:51.524266 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:51.524733 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:51.524906 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:51.528915 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:51.528921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:51.528925 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:51.528963 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:51.532080 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:51.532945 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:51.532952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:51.532956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:51.533094 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:51.533603 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:51.533614 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:51.533616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:51.533899 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:51.534086 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:51.534885 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 18:20:51.535151 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":347,"threads_started":1}
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
I20260807 18:20:51.536746 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:51.515555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:42559 (local address 127.29.156.193:60659)
0807 18:20:51.516682 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:51.516748 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:51.519130 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:51.520030 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:51.520039 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:51.520645 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:51.521392 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:51.521411 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:51.522601 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:51.522607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:51.524099 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:51.524112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:51.525038 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:51.525047 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:51.525297 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:51.526092 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:51.526116 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:51.528646 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:51.532258 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:51.532269 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:51.532289 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:51.532784 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:51.533237 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:51.533242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:51.533246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:51.533457 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:51.534059 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:51.534068 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:51.534448 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 18:20:51.534762 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 18:20:51.535357 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":459,"threads_started":1}
I20260807 18:20:51.538651 30683 heartbeater.cc:347] Connected to a master server at 127.29.156.254:42559
I20260807 18:20:51.539049 30683 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
I20260807 18:20:51.539943 30683 heartbeater.cc:511] Master 127.29.156.254:42559 requested a full tablet report, sending...
I20260807 18:20:51.542287 30409 ts_manager.cc:194] Registered new tserver with Master: 94f5e7f4b8074f4d86a157881b57d2a8 (127.29.156.193:42855)
I20260807 18:20:51.543885 30409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:60659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:20:51.568616 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:51.961081 30691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:51.961405 30691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:51.961561 30691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:51.969765 30691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:51.969900 30691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:51.970131 30691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:20:51.981813 30691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:51.983382 30691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:51.984938 30691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:51.999964 30697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:52.000144 30702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:52.000855 30698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:52.001959 30691 server_base.cc:1034] running on GCE node
I20260807 18:20:52.002812 30691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:52.004009 30691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:52.005275 30691 hybrid_clock.cc:648] HybridClock initialized: now 1786126852005174 us; error 57 us; skew 500 ppm
I20260807 18:20:52.005668 30691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:52.009704 30691 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:20:52.011802 30691 webserver.cc:460] Webserver started at http://127.29.156.194:38409/ using document root <none> and password file <none>
I20260807 18:20:52.012557 30691 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:52.012720 30691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:52.013072 30691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:52.016445 30691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "e68ff546689d4418ab667bfd77410ba1"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "9ce1b5562f4953376e23c81a0d6e2546"
server_key_iv: "b694d9b9b69ab906e3fc34c530737c6e"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.017343 30691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "e68ff546689d4418ab667bfd77410ba1"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "9ce1b5562f4953376e23c81a0d6e2546"
server_key_iv: "b694d9b9b69ab906e3fc34c530737c6e"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.023433 30691 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:20:52.027382 30713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.028975 30691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:20:52.029191 30691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "e68ff546689d4418ab667bfd77410ba1"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "9ce1b5562f4953376e23c81a0d6e2546"
server_key_iv: "b694d9b9b69ab906e3fc34c530737c6e"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.029484 30691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:52.051414 30691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:52.056438 30691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:52.056900 30691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:52.058111 30691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:52.059917 30691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:52.060009 30691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.060128 30691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:52.060204 30691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.113551 30691 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:37939
I20260807 18:20:52.113668 30876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:37939 every 8 connection(s)
I20260807 18:20:52.115114 30691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:20:52.123634 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30691
I20260807 18:20:52.123924 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:20:52.124555 30323 external_mini_cluster.cc:1428] Setting key b6cb9f7c0563791d4409e23027440f6c
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:52.143198 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.118177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:53229 (local address 127.29.156.254:42559)
0807 18:20:52.118766 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:52.118771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:52.119939 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 18:20:52.122229 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:52.122235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:52.122327 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:52.122455 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:52.124877 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.125724 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.127884 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.128870 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.129069 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.134210 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:52.134220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:52.134225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:52.134296 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:52.138703 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.139631 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.139640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.139646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.139851 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.140440 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.140448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.140454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.140953 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:52.141126 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:52.142353 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0807 18:20:52.142676 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":369,"threads_started":1}
I20260807 18:20:52.143833 30882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.118751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:42559 (local address 127.29.156.194:53229)
0807 18:20:52.119737 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:52.119804 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:52.121926 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:52.122714 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:52.122722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:52.123635 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:52.124637 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:52.124666 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.125953 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.125958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:52.127712 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:52.127725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.129155 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.129160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:52.129386 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.130296 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:52.130325 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:52.133734 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:52.138895 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:52.138903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:52.138918 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:52.139450 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:52.140023 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:52.140027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:52.140029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:52.140292 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:52.141106 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:52.141112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:52.141607 (+ 495us) client_negotiation.cc:769] Sending connection context
0807 18:20:52.141972 (+ 365us) client_negotiation.cc:240] Negotiation successful
0807 18:20:52.142542 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":396,"threads_started":1}
I20260807 18:20:52.146026 30877 heartbeater.cc:347] Connected to a master server at 127.29.156.254:42559
I20260807 18:20:52.146418 30877 heartbeater.cc:464] Registering TS with master...
I20260807 18:20:52.147430 30877 heartbeater.cc:511] Master 127.29.156.254:42559 requested a full tablet report, sending...
I20260807 18:20:52.149309 30409 ts_manager.cc:194] Registered new tserver with Master: e68ff546689d4418ab667bfd77410ba1 (127.29.156.194:37939)
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
I20260807 18:20:52.150298 30409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:53229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:20:52.184051 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:20:52.547019 30683 heartbeater.cc:503] Master 127.29.156.254:42559 was elected leader, sending a full tablet report...
W20260807 18:20:52.610365 30886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:52.610669 30886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:52.610774 30886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:52.619064 30886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:52.619192 30886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:52.619313 30886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:20:52.630559 30886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:42559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:52.632077 30886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:52.633400 30886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:52.648804 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:52.648759 30894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:52.648850 30898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:52.650266 30886 server_base.cc:1034] running on GCE node
I20260807 18:20:52.650938 30886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:52.652252 30886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:52.653551 30886 hybrid_clock.cc:648] HybridClock initialized: now 1786126852653481 us; error 54 us; skew 500 ppm
I20260807 18:20:52.653957 30886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:52.657987 30886 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:20:52.659791 30886 webserver.cc:460] Webserver started at http://127.29.156.195:43117/ using document root <none> and password file <none>
I20260807 18:20:52.660491 30886 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:52.660600 30886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:52.660992 30886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:52.664175 30886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "b2037b23163b4d0aa3e0af4e70db5393"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "46f0b84b82af0ff697a55c373346f066"
server_key_iv: "a237c4aca804b326d25a0edbba6253da"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.665025 30886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "b2037b23163b4d0aa3e0af4e70db5393"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "46f0b84b82af0ff697a55c373346f066"
server_key_iv: "a237c4aca804b326d25a0edbba6253da"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.670746 30886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:20:52.674970 30906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.676450 30886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:20:52.676600 30886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "b2037b23163b4d0aa3e0af4e70db5393"
format_stamp: "Formatted at 2026-08-07 18:20:52 on dist-test-slave-sb2z"
server_key: "46f0b84b82af0ff697a55c373346f066"
server_key_iv: "a237c4aca804b326d25a0edbba6253da"
server_key_version: "encryptionkey@0"
I20260807 18:20:52.676789 30886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:52.700259 30886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:52.705226 30886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:52.705572 30886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:52.706856 30886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:52.708734 30886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:52.708817 30886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.708906 30886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:52.708961 30886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:52.764477 30886 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:43253
I20260807 18:20:52.764544 31077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:43253 every 8 connection(s)
I20260807 18:20:52.766134 30886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:20:52.769943 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30886
I20260807 18:20:52.770205 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:20:52.770771 30323 external_mini_cluster.cc:1428] Setting key 6cda9261a88525dcbd8f761d196cda4c
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:52.791915 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.769328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:53953 (local address 127.29.156.254:42559)
0807 18:20:52.770105 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:52.770112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:52.771796 (+ 1684us) server_negotiation.cc:407] Connection header received
0807 18:20:52.774301 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:52.774305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:52.774396 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:52.774511 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:52.776513 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.777495 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.779645 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.780301 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.780426 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.784898 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:52.784904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:52.784907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:52.784943 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:52.788317 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.789107 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.789111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.789114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.789241 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.789760 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.789764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.789766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.790057 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:52.790186 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:52.791192 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 18:20:52.791390 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":355,"threads_started":1}
I20260807 18:20:52.793092 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.770207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:42559 (local address 127.29.156.195:53953)
0807 18:20:52.771479 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:52.771566 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:52.773893 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:52.774827 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:52.774840 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:52.775615 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:52.776297 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:52.776319 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.777710 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.777715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:52.779427 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:52.779442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.780562 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.780568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:52.780945 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.781779 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:52.781804 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:52.784620 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:52.788460 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:52.788469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:52.788492 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:52.788969 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:52.789377 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:52.789381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:52.789384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:52.789620 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:52.790238 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:52.790245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:52.790632 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 18:20:52.790905 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 18:20:52.791632 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":547,"threads_started":1}
I20260807 18:20:52.794837 31078 heartbeater.cc:347] Connected to a master server at 127.29.156.254:42559
I20260807 18:20:52.795230 31078 heartbeater.cc:464] Registering TS with master...
I20260807 18:20:52.796168 31078 heartbeater.cc:511] Master 127.29.156.254:42559 requested a full tablet report, sending...
I20260807 18:20:52.797742 30409 ts_manager.cc:194] Registered new tserver with Master: b2037b23163b4d0aa3e0af4e70db5393 (127.29.156.195:43253)
I20260807 18:20:52.798982 30409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:53953
I20260807 18:20:52.801378 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:20:52.865386 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.845673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.29.156.193:42855)
0807 18:20:52.846499 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:52.846508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:52.846539 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:20:52.846674 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:52.846681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:52.846904 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:52.847129 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:52.848888 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.850472 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.852128 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.852737 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.852849 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.857895 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:52.857911 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:52.857928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:52.857970 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:52.862031 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.862798 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.862806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.862809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.862946 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.863363 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.863366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.863369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.863924 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:52.864229 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:52.864575 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 18:20:52.864873 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":490,"threads_started":1}
W20260807 18:20:52.866925 30558 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35176
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:20:52 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:20:52.926774 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.909599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.29.156.193:42855)
0807 18:20:52.909928 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:52.909939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:52.910440 (+ 501us) server_negotiation.cc:407] Connection header received
0807 18:20:52.910620 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:52.910625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:52.910695 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:52.910787 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:52.912358 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.913110 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.914638 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.915144 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.915260 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.919824 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:52.919829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:52.919833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:52.919878 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:52.923414 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.924295 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.924299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.924302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.924472 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.925026 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.925031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.925033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.925337 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:52.925622 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:52.926109 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 18:20:52.926318 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 18:20:52.928741 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30503
I20260807 18:20:52.944173 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:52.930013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.29.156.193:42855)
0807 18:20:52.930288 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:52.930297 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:52.930605 (+ 308us) server_negotiation.cc:407] Connection header received
0807 18:20:52.930792 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:52.930803 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:52.930893 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:52.931010 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:52.932383 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.933338 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.934832 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:52.935440 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:52.935625 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:52.938084 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:52.938090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:52.938095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:52.938133 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:52.941241 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.941999 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.942006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.942009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.942212 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:52.942661 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:52.942666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:52.942669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:52.942966 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:52.943232 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:52.943565 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:20:52.943806 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:20:52.944994 30677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:53.031761 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30503
I20260807 18:20:53.055065 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30691
Aug 07 18:20:53 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:20:53.072012 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:53.056210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.29.156.194:37939)
0807 18:20:53.056852 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:53.056859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:53.056888 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:20:53.056951 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:53.056957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:53.057168 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:53.057299 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:53.058705 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:53.060008 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:53.061331 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:53.061898 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:53.061994 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:53.066238 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:53.066249 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:53.066265 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:53.066306 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:53.069271 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:53.069957 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:53.069964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:53.069967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:53.070091 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:53.070457 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:53.070461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:53.070463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:53.070822 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:53.071048 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:53.071324 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:20:53.071594 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":330,"threads_started":1}
I20260807 18:20:53.073346 30871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:53.159273 30877 heartbeater.cc:503] Master 127.29.156.254:42559 was elected leader, sending a full tablet report...
I20260807 18:20:53.163838 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30691
I20260807 18:20:53.188755 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30886
Aug 07 18:20:53 dist-test-slave-sb2z krb5kdc[30336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:20:53.207608 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:53.190197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.29.156.195:43253)
0807 18:20:53.190881 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:53.190888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:53.190921 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:20:53.190985 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:53.190991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:53.191211 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:53.191446 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:53.192994 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:53.194050 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:53.195391 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:53.196041 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:53.196144 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:53.200853 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:53.200866 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:53.200903 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:53.200955 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:53.204389 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:53.205165 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:53.205173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:53.205175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:53.205316 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:53.205752 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:53.205756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:53.205758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:53.206186 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:53.206448 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:53.206811 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 18:20:53.207155 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":391,"threads_started":1}
I20260807 18:20:53.209067 31070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:53.298305 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30886
I20260807 18:20:53.326628 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30360
I20260807 18:20:53.328296 30450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:53.464798 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30360
2026-08-07T18:20:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4841 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:53 dist-test-slave-sb2z krb5kdc[31110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:53 dist-test-slave-sb2z krb5kdc[31110](info): set up 2 sockets
Aug 07 18:20:53 dist-test-slave-sb2z krb5kdc[31110](info): commencing operation
krb5kdc: starting...
W20260807 18:20:54.753167 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.010s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:20:54 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:20:54.979895 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41627
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:55.324124 31129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:55.324568 31129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:55.324832 31129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:55.332903 31129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:55.333067 31129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:55.333181 31129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:55.333295 31129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:55.333386 31129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:55.343761 31129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:55.345207 31129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:55.346612 31129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:55.359457 31136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:55.359777 31140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:55.359901 31135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:55.360801 31129 server_base.cc:1034] running on GCE node
I20260807 18:20:55.361711 31129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:55.363163 31129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:55.364481 31129 hybrid_clock.cc:648] HybridClock initialized: now 1786126855364420 us; error 26 us; skew 500 ppm
I20260807 18:20:55.364928 31129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:55 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126855, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:55.374508 31129 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:20:55.376446 31129 webserver.cc:460] Webserver started at http://127.29.156.254:32875/ using document root <none> and password file <none>
I20260807 18:20:55.377277 31129 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:55.377441 31129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:55.377833 31129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:55.381402 31129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "1c0468ec55c9431684ff227154d7129d"
format_stamp: "Formatted at 2026-08-07 18:20:55 on dist-test-slave-sb2z"
server_key: "51b6b313e47b07606b99434fc0150528"
server_key_iv: "31511a3a70f089d20ef38deebd8660bc"
server_key_version: "encryptionkey@0"
I20260807 18:20:55.382302 31129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "1c0468ec55c9431684ff227154d7129d"
format_stamp: "Formatted at 2026-08-07 18:20:55 on dist-test-slave-sb2z"
server_key: "51b6b313e47b07606b99434fc0150528"
server_key_iv: "31511a3a70f089d20ef38deebd8660bc"
server_key_version: "encryptionkey@0"
I20260807 18:20:55.388268 31129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:20:55.392287 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:55.393736 31129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 18:20:55.393950 31129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "1c0468ec55c9431684ff227154d7129d"
format_stamp: "Formatted at 2026-08-07 18:20:55 on dist-test-slave-sb2z"
server_key: "51b6b313e47b07606b99434fc0150528"
server_key_iv: "31511a3a70f089d20ef38deebd8660bc"
server_key_version: "encryptionkey@0"
I20260807 18:20:55.394240 31129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:55.436834 31129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:55.441797 31129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:55.442142 31129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:55.462826 31129 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:41627
I20260807 18:20:55.462838 31224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:41627 every 8 connection(s)
I20260807 18:20:55.464298 31129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:55.468257 31225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:55.473450 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 31129
I20260807 18:20:55.473699 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:20:55.474398 30323 external_mini_cluster.cc:1428] Setting key 7b9c9939ce512d4a41b36965ea3f2f02
I20260807 18:20:55.479574 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d: Bootstrap starting.
I20260807 18:20:55.483503 31225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:55.484930 31225 log.cc:824] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:55.488912 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d: No bootstrap required, opened a new log
Aug 07 18:20:55 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:55.497723 31225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } }
I20260807 18:20:55.498236 31225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:55.498397 31225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0468ec55c9431684ff227154d7129d, State: Initialized, Role: FOLLOWER
I20260807 18:20:55.499200 31225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } }
I20260807 18:20:55.499464 31225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:55.499673 31225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:55.499889 31225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:55.499927 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:55.476720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.29.156.254:41627)
0807 18:20:55.477688 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:55.477701 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:55.477744 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:20:55.479472 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:55.479495 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:55.479995 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:55.480319 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:55.482349 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:55.483783 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:55.485601 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:55.486723 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:55.486858 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:55.491881 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:55.491892 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:55.491911 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:55.491955 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:55.495228 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:55.496078 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:55.496088 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:55.496097 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:55.496293 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:55.496727 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:55.496731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:55.496733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:55.497315 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:55.497646 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:55.498153 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 18:20:55.498666 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":473,"threads_started":1}
I20260807 18:20:55.502499 31225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } }
I20260807 18:20:55.503088 31225 leader_election.cc:302] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c0468ec55c9431684ff227154d7129d; no voters:
I20260807 18:20:55.503784 31225 leader_election.cc:288] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:55.503937 31233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:55.504678 31233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [term 1 LEADER]: Becoming Leader. State: Replica: 1c0468ec55c9431684ff227154d7129d, State: Running, Role: LEADER
I20260807 18:20:55.505244 31233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } }
I20260807 18:20:55.506011 31225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:55.508136 31238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0468ec55c9431684ff227154d7129d. Latest consensus state: current_term: 1 leader_uuid: "1c0468ec55c9431684ff227154d7129d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } } }
I20260807 18:20:55.509284 31238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:55.508181 31234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c0468ec55c9431684ff227154d7129d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0468ec55c9431684ff227154d7129d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41627 } } }
I20260807 18:20:55.510100 31234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:55.510334 31247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:55.518539 31247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:55.530521 31247 catalog_manager.cc:1347] Generated new cluster ID: 0b618e77f0924de6957e6997de57cc8c
I20260807 18:20:55.530658 31247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:55.541941 31247 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:55.543447 31247 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:55.551117 31247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c0468ec55c9431684ff227154d7129d: Generated new TSK 0
I20260807 18:20:55.552060 31247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:20:55.619132 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:55.996568 31266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:55.996896 31266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:55.997083 31266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:56.005482 31266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:56.005668 31266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:56.005877 31266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:20:56.016747 31266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:56.018210 31266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:56.019759 31266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:56.034204 31274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:56.034173 31275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:56.034822 31278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:56.035157 31266 server_base.cc:1034] running on GCE node
I20260807 18:20:56.035921 31266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:56.037063 31266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:56.038300 31266 hybrid_clock.cc:648] HybridClock initialized: now 1786126856038228 us; error 58 us; skew 500 ppm
I20260807 18:20:56.038628 31266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:56 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126856, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:56.042668 31266 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:20:56.044528 31266 webserver.cc:460] Webserver started at http://127.29.156.193:46231/ using document root <none> and password file <none>
I20260807 18:20:56.045367 31266 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:56.045545 31266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:56.045915 31266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:56.049300 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "79f4adbdccf568e9931a35b1fe61373b"
server_key_iv: "3a6ba70661bfe406b9bb30c6ff1a6a2a"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.050298 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "79f4adbdccf568e9931a35b1fe61373b"
server_key_iv: "3a6ba70661bfe406b9bb30c6ff1a6a2a"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.056190 31266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:20:56.060240 31288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.061692 31266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:20:56.061904 31266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "79f4adbdccf568e9931a35b1fe61373b"
server_key_iv: "3a6ba70661bfe406b9bb30c6ff1a6a2a"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.062160 31266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:56.089365 31266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:56.095052 31266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:56.095412 31266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:56.096786 31266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:56.098371 31266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:56.098496 31266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.098685 31266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:56.098778 31266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.138036 31266 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:44991
I20260807 18:20:56.138152 31452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:44991 every 8 connection(s)
I20260807 18:20:56.139622 31266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:20:56.145905 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 31266
I20260807 18:20:56.146131 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:20:56.146776 30323 external_mini_cluster.cc:1428] Setting key 53de8797e6df42c3b9301f9bd44b1d11
Aug 07 18:20:56 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126856, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:56.162674 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:56.142579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:41853 (local address 127.29.156.254:41627)
0807 18:20:56.143086 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:56.143091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:56.144198 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 18:20:56.146496 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:56.146503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:56.146603 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:56.146760 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:56.148806 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.149640 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.151437 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.152023 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.152204 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:56.156435 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:56.156440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:56.156444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:56.156475 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:56.159262 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:56.160135 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:56.160139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:56.160142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:56.160266 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:56.160752 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:56.160764 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:56.160767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:56.161080 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:56.161260 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:56.162044 (+ 784us) server_negotiation.cc:299] Negotiation successful
0807 18:20:56.162278 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":247,"threads_started":1}
I20260807 18:20:56.163870 31456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:56.143148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41627 (local address 127.29.156.193:41853)
0807 18:20:56.143995 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:56.144051 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:56.146154 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:56.147019 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:56.147028 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:56.147721 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:56.148597 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:56.148626 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.149821 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.149827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:56.151223 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:56.151237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.152333 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.152341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:56.152604 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:56.153404 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:56.153429 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:56.156183 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:56.159428 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:56.159437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:56.159454 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:56.159977 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:56.160395 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:56.160400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:56.160404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:56.160617 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:56.161228 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:56.161236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:56.161612 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 18:20:56.161905 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 18:20:56.162496 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":376,"threads_started":1}
I20260807 18:20:56.165797 31453 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41627
I20260807 18:20:56.166281 31453 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
I20260807 18:20:56.167207 31453 heartbeater.cc:511] Master 127.29.156.254:41627 requested a full tablet report, sending...
I20260807 18:20:56.169728 31179 ts_manager.cc:194] Registered new tserver with Master: ba9dc30a856a4a628cfe62d1f41aa5c9 (127.29.156.193:44991)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
I20260807 18:20:56.171682 31179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:41853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:20:56.200521 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:56.549746 31460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:56.550060 31460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:56.550241 31460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:56.557574 31460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:56.557677 31460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:56.557904 31460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:20:56.568311 31460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:56.569643 31460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:56.571029 31460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:56.585068 31465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:56.585026 31467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:56.585552 31470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:56.586087 31460 server_base.cc:1034] running on GCE node
I20260807 18:20:56.586716 31460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:56.587759 31460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:56.589036 31460 hybrid_clock.cc:648] HybridClock initialized: now 1786126856588981 us; error 45 us; skew 500 ppm
I20260807 18:20:56.589419 31460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:56 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126856, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:56.593233 31460 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:20:56.594914 31460 webserver.cc:460] Webserver started at http://127.29.156.194:33917/ using document root <none> and password file <none>
I20260807 18:20:56.595736 31460 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:56.595911 31460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:56.596251 31460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:56.599488 31460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "e4f0b47924a5472db1668d8e66c776b0"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "43e45477ea93e4b779d60c5c7f05a9e1"
server_key_iv: "1be428ddff3f2a1ce2067f5612f4ad75"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.600430 31460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "e4f0b47924a5472db1668d8e66c776b0"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "43e45477ea93e4b779d60c5c7f05a9e1"
server_key_iv: "1be428ddff3f2a1ce2067f5612f4ad75"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.606142 31460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:20:56.610091 31486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.611661 31460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:20:56.611843 31460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "e4f0b47924a5472db1668d8e66c776b0"
format_stamp: "Formatted at 2026-08-07 18:20:56 on dist-test-slave-sb2z"
server_key: "43e45477ea93e4b779d60c5c7f05a9e1"
server_key_iv: "1be428ddff3f2a1ce2067f5612f4ad75"
server_key_version: "encryptionkey@0"
I20260807 18:20:56.612071 31460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:56.627731 31460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:56.632859 31460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:56.633208 31460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:56.634433 31460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:56.636104 31460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:56.636186 31460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.636372 31460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:56.636461 31460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:56.673974 31460 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:42329
I20260807 18:20:56.674028 31656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:42329 every 8 connection(s)
I20260807 18:20:56.675613 31460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:20:56.685081 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 31460
I20260807 18:20:56.685333 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:20:56.685912 30323 external_mini_cluster.cc:1428] Setting key 69ce7e5dc0b9ce9d53fc2676552f83cb
Aug 07 18:20:56 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126856, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:56.700145 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:56.678641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:49833 (local address 127.29.156.254:41627)
0807 18:20:56.679123 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:56.679131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:56.680443 (+ 1312us) server_negotiation.cc:407] Connection header received
0807 18:20:56.682341 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:56.682346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:56.682420 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:56.682541 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:56.684349 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.685174 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.687474 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.688335 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.688505 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:56.693278 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:56.693284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:56.693287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:56.693324 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:56.696407 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:56.697262 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:56.697270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:56.697279 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:56.697419 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:56.698028 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:56.698036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:56.698045 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:56.698351 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:56.698538 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:56.699394 (+ 856us) server_negotiation.cc:299] Negotiation successful
0807 18:20:56.699683 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":353,"thread_start_us":232,"threads_started":1}
I20260807 18:20:56.701246 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:56.679224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41627 (local address 127.29.156.194:49833)
0807 18:20:56.680258 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:56.680300 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:56.682047 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:56.682768 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:56.682776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:56.683385 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:56.684165 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:56.684185 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.685419 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.685437 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:56.687279 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:56.687292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:56.688652 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:56.688660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:56.688994 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:56.689815 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:56.689851 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:56.692986 (+ 3135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:56.696583 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:56.696593 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:56.696613 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:56.697112 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:56.697557 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:56.697563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:56.697567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:56.697887 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:56.698512 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:56.698521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:56.698982 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 18:20:56.699280 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 18:20:56.699913 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":543,"threads_started":1}
I20260807 18:20:56.703001 31658 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41627
I20260807 18:20:56.703346 31658 heartbeater.cc:464] Registering TS with master...
I20260807 18:20:56.704289 31658 heartbeater.cc:511] Master 127.29.156.254:41627 requested a full tablet report, sending...
I20260807 18:20:56.706187 31179 ts_manager.cc:194] Registered new tserver with Master: e4f0b47924a5472db1668d8e66c776b0 (127.29.156.194:42329)
I20260807 18:20:56.707448 31179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:49833
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:20:56.745121 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:57.100244 31667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:57.100543 31667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:57.100652 31667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:57.108127 31667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:57.108242 31667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:57.108358 31667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:20:57.118777 31667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:57.120165 31667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:57.121533 31667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:57.136087 31682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:57.136283 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:57.136179 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:57.137176 31667 server_base.cc:1034] running on GCE node
I20260807 18:20:57.138130 31667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:57.139554 31667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:57.140841 31667 hybrid_clock.cc:648] HybridClock initialized: now 1786126857140765 us; error 56 us; skew 500 ppm
I20260807 18:20:57.141284 31667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:57.145716 31667 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:20:57.147325 31667 webserver.cc:460] Webserver started at http://127.29.156.195:33951/ using document root <none> and password file <none>
I20260807 18:20:57.148047 31667 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:57.148161 31667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:57.148571 31667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:57.151476 31667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "aa4d5d9af0a442229fad7bd504d94484"
format_stamp: "Formatted at 2026-08-07 18:20:57 on dist-test-slave-sb2z"
server_key: "25b82380eec766f8568d1e3fe73450ef"
server_key_iv: "046a06b762c9128d5099b326854dcf35"
server_key_version: "encryptionkey@0"
I20260807 18:20:57.152360 31667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "aa4d5d9af0a442229fad7bd504d94484"
format_stamp: "Formatted at 2026-08-07 18:20:57 on dist-test-slave-sb2z"
server_key: "25b82380eec766f8568d1e3fe73450ef"
server_key_iv: "046a06b762c9128d5099b326854dcf35"
server_key_version: "encryptionkey@0"
I20260807 18:20:57.157764 31667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:20:57.161367 31691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:57.162811 31667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:20:57.163005 31667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "aa4d5d9af0a442229fad7bd504d94484"
format_stamp: "Formatted at 2026-08-07 18:20:57 on dist-test-slave-sb2z"
server_key: "25b82380eec766f8568d1e3fe73450ef"
server_key_iv: "046a06b762c9128d5099b326854dcf35"
server_key_version: "encryptionkey@0"
I20260807 18:20:57.163266 31667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:57.174791 31453 heartbeater.cc:503] Master 127.29.156.254:41627 was elected leader, sending a full tablet report...
I20260807 18:20:57.193874 31667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:57.198562 31667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:57.198894 31667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:57.200136 31667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:20:57.201668 31667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:20:57.201784 31667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:57.201956 31667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:20:57.202045 31667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:57.255735 31667 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:37287
I20260807 18:20:57.255858 31855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:37287 every 8 connection(s)
I20260807 18:20:57.257462 31667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:20:57.260638 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 31667
I20260807 18:20:57.260913 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:20:57.261549 30323 external_mini_cluster.cc:1428] Setting key 0f9209aac4ed4cd27ca73415cd1e7ac5
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:20:57.282809 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.260975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:54361 (local address 127.29.156.254:41627)
0807 18:20:57.261612 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.261619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.263070 (+ 1451us) server_negotiation.cc:407] Connection header received
0807 18:20:57.265437 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.265449 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.265536 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.265674 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.267450 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.268479 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.270263 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.270696 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.270851 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.275237 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.275243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.275247 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.275283 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.279234 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.280174 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.280178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.280181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.280304 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.280790 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.280795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.280802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.281101 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.281282 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.282107 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.282342 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":296,"threads_started":1}
I20260807 18:20:57.284027 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.261755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41627 (local address 127.29.156.195:54361)
0807 18:20:57.262900 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.262947 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.265154 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.265950 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.265959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.266621 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:57.267281 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.267298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.268677 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.268682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.270087 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.270100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.271013 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.271023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.271260 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.272208 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:57.272231 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:57.274984 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:57.279407 (+ 4423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:57.279419 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:57.279440 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:57.280007 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:57.280433 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:57.280440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:57.280444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:57.280653 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:57.281249 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:57.281257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:57.281690 (+ 433us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.282003 (+ 313us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.282566 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":590,"threads_started":1}
I20260807 18:20:57.285785 31856 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41627
I20260807 18:20:57.286087 31856 heartbeater.cc:464] Registering TS with master...
I20260807 18:20:57.286856 31856 heartbeater.cc:511] Master 127.29.156.254:41627 requested a full tablet report, sending...
I20260807 18:20:57.288450 31179 ts_manager.cc:194] Registered new tserver with Master: aa4d5d9af0a442229fad7bd504d94484 (127.29.156.195:37287)
I20260807 18:20:57.289377 31179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:54361
I20260807 18:20:57.291154 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:20:57.313313 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.298196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.29.156.254:41627)
0807 18:20:57.298389 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.298394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.299084 (+ 690us) server_negotiation.cc:407] Connection header received
0807 18:20:57.299303 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.299308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.299411 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.299521 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.301254 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.302093 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.303566 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.304086 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.304236 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.306770 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.306776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.306779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.306843 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.310020 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.310792 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.310800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.310804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.310958 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.311406 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.311414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.311420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.311856 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.312173 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.312619 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.312905 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:20:57.321584 31179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:20:57.331107 31179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:20:57.375787 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.345952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.29.156.193:44991)
0807 18:20:57.363328 (+ 17376us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.363344 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.363389 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:20:57.363574 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.363589 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.363977 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.364310 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:20:57.365673 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.369012 (+ 3339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.372374 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.374026 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.374231 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.374408 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.375198 (+ 790us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.375372 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17163,"thread_start_us":16955,"threads_started":1}
I20260807 18:20:57.377039 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.345192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:44991 (local address 127.0.0.1:50228)
0807 18:20:57.360021 (+ 14829us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.360054 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.360243 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.364696 (+ 4453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.364710 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.364758 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:20:57.365348 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.365366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.369208 (+ 3842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.369214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.372166 (+ 2952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.372189 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.374280 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.374288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.374872 (+ 584us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.374945 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.375113 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.376511 (+ 1398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14602,"mutex_wait_us":215,"thread_start_us":21628,"threads_started":1}
I20260807 18:20:57.377883 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.347693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.29.156.194:42329)
0807 18:20:57.368254 (+ 20561us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.368266 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.368809 (+ 543us) server_negotiation.cc:407] Connection header received
0807 18:20:57.368953 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.368965 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.369319 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.369558 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:20:57.370469 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.371953 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.374771 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.376264 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.376435 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.376622 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.377302 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.377484 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20335,"thread_start_us":20148,"threads_started":1}
I20260807 18:20:57.378227 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.347228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42329 (local address 127.0.0.1:41356)
0807 18:20:57.368465 (+ 21237us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.368492 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.368657 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.369810 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.369814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.369844 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:20:57.370258 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.370268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.372101 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.372112 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.374550 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.374562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.376830 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.376836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.376966 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.376993 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.377121 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.377255 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20282,"mutex_wait_us":1,"thread_start_us":13145,"threads_started":1}
I20260807 18:20:57.378373 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.344739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:37287 (local address 127.0.0.1:43188)
0807 18:20:57.346783 (+ 2044us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.346831 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.347018 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.370585 (+ 23567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.370591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.370621 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:20:57.371092 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.371106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.373043 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.373050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.375286 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.375301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.377370 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.377381 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.377625 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.377681 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.377770 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.377929 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1782,"mutex_wait_us":306,"thread_start_us":1603,"threads_started":1}
I20260807 18:20:57.379740 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.345079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.29.156.195:37287)
0807 18:20:57.369897 (+ 24818us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.369908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.369946 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:20:57.370062 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.370072 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.370357 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.370593 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:20:57.371318 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.372853 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.375480 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.377044 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.378909 (+ 1865us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.379092 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.379168 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.379320 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24630,"thread_start_us":24451,"threads_started":1}
I20260807 18:20:57.382055 31359 tablet_service.cc:1467] Processing CreateTablet for tablet add7c11b40c147f9858570e2f130c417 (DEFAULT_TABLE table=test-table [id=c82852dac5c6481183c52828100eab33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:20:57.382591 31555 tablet_service.cc:1467] Processing CreateTablet for tablet add7c11b40c147f9858570e2f130c417 (DEFAULT_TABLE table=test-table [id=c82852dac5c6481183c52828100eab33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:20:57.382882 31756 tablet_service.cc:1467] Processing CreateTablet for tablet add7c11b40c147f9858570e2f130c417 (DEFAULT_TABLE table=test-table [id=c82852dac5c6481183c52828100eab33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:20:57.383872 31359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add7c11b40c147f9858570e2f130c417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:57.384112 31555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add7c11b40c147f9858570e2f130c417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:57.384418 31756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add7c11b40c147f9858570e2f130c417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:57.397883 31881 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Bootstrap starting.
I20260807 18:20:57.401218 31882 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Bootstrap starting.
I20260807 18:20:57.402163 31883 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Bootstrap starting.
I20260807 18:20:57.403029 31881 tablet_bootstrap.cc:654] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:57.404873 31881 log.cc:824] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:57.406700 31882 tablet_bootstrap.cc:654] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:57.407472 31883 tablet_bootstrap.cc:654] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:57.408293 31882 log.cc:824] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:57.409015 31883 log.cc:824] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:57.409279 31881 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: No bootstrap required, opened a new log
I20260807 18:20:57.409926 31881 ts_tablet_manager.cc:1397] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20260807 18:20:57.415210 31882 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: No bootstrap required, opened a new log
I20260807 18:20:57.415210 31883 tablet_bootstrap.cc:492] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: No bootstrap required, opened a new log
I20260807 18:20:57.415660 31882 ts_tablet_manager.cc:1397] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260807 18:20:57.415694 31883 ts_tablet_manager.cc:1397] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260807 18:20:57.419947 31881 raft_consensus.cc:348] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.420420 31881 raft_consensus.cc:374] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:57.420603 31881 raft_consensus.cc:729] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4d5d9af0a442229fad7bd504d94484, State: Initialized, Role: FOLLOWER
I20260807 18:20:57.421343 31881 consensus_queue.cc:260] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.422725 31856 heartbeater.cc:503] Master 127.29.156.254:41627 was elected leader, sending a full tablet report...
I20260807 18:20:57.423604 31881 ts_tablet_manager.cc:1428] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:20:57.429297 31883 raft_consensus.cc:348] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.429347 31882 raft_consensus.cc:348] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.429890 31883 raft_consensus.cc:374] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:57.429900 31882 raft_consensus.cc:374] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:57.430038 31883 raft_consensus.cc:729] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9dc30a856a4a628cfe62d1f41aa5c9, State: Initialized, Role: FOLLOWER
I20260807 18:20:57.430037 31882 raft_consensus.cc:729] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f0b47924a5472db1668d8e66c776b0, State: Initialized, Role: FOLLOWER
I20260807 18:20:57.430748 31882 consensus_queue.cc:260] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.430725 31883 consensus_queue.cc:260] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.432271 31658 heartbeater.cc:503] Master 127.29.156.254:41627 was elected leader, sending a full tablet report...
I20260807 18:20:57.432931 31882 ts_tablet_manager.cc:1428] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260807 18:20:57.432910 31883 ts_tablet_manager.cc:1428] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 18:20:57.457239 31889 raft_consensus.cc:482] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:20:57.457568 31889 raft_consensus.cc:504] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.459403 31889 leader_election.cc:288] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4d5d9af0a442229fad7bd504d94484 (127.29.156.195:37287), ba9dc30a856a4a628cfe62d1f41aa5c9 (127.29.156.193:44991)
I20260807 18:20:57.469879 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.460060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:41609 (local address 127.29.156.193:44991)
0807 18:20:57.460276 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.460283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.461481 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 18:20:57.461701 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.461707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.461820 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.461920 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:20:57.463668 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.464539 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.467456 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.468644 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.468893 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.469003 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.469408 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.469557 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:20:57.470772 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.460087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:43041 (local address 127.29.156.195:37287)
0807 18:20:57.460286 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.460293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.461481 (+ 1188us) server_negotiation.cc:407] Connection header received
0807 18:20:57.461701 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.461707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.461797 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.461920 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:20:57.463127 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.464522 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.468032 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.469439 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.469580 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.469659 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.470351 (+ 692us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.470505 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:20:57.471326 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.459912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:44991 (local address 127.29.156.194:41609)
0807 18:20:57.461323 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.461350 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.461538 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.462049 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.462054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.462092 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:20:57.462939 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.462947 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.464707 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.464712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.466896 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.466913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.468975 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.468983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.469153 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.469180 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.470357 (+ 1177us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.470514 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":96,"spinlock_wait_cycles":17664,"thread_start_us":849,"threads_started":1}
I20260807 18:20:57.471755 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.459954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:37287 (local address 127.29.156.194:43041)
0807 18:20:57.461355 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.461371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:57.461538 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:57.462059 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:57.462063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:57.462095 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:20:57.462915 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.462943 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.464708 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.464715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.467774 (+ 3059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:57.467792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.469912 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.469922 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:57.470150 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.470187 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:20:57.470997 (+ 810us) client_negotiation.cc:240] Negotiation successful
0807 18:20:57.471171 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1296,"mutex_wait_us":63,"spinlock_wait_cycles":26496,"thread_start_us":646,"threads_started":1}
I20260807 18:20:57.472750 31784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add7c11b40c147f9858570e2f130c417" candidate_uuid: "e4f0b47924a5472db1668d8e66c776b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4d5d9af0a442229fad7bd504d94484" is_pre_election: true
I20260807 18:20:57.472731 31385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add7c11b40c147f9858570e2f130c417" candidate_uuid: "e4f0b47924a5472db1668d8e66c776b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" is_pre_election: true
I20260807 18:20:57.473440 31784 raft_consensus.cc:2454] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f0b47924a5472db1668d8e66c776b0 in term 0.
I20260807 18:20:57.473510 31385 raft_consensus.cc:2454] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f0b47924a5472db1668d8e66c776b0 in term 0.
I20260807 18:20:57.474323 31491 leader_election.cc:302] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba9dc30a856a4a628cfe62d1f41aa5c9, e4f0b47924a5472db1668d8e66c776b0; no voters:
I20260807 18:20:57.474802 31889 raft_consensus.cc:2781] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:20:57.474975 31889 raft_consensus.cc:482] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:20:57.475086 31889 raft_consensus.cc:3037] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:57.478595 31889 raft_consensus.cc:504] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.479408 31889 leader_election.cc:288] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [CANDIDATE]: Term 1 election: Requested vote from peers aa4d5d9af0a442229fad7bd504d94484 (127.29.156.195:37287), ba9dc30a856a4a628cfe62d1f41aa5c9 (127.29.156.193:44991)
I20260807 18:20:57.479965 31784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add7c11b40c147f9858570e2f130c417" candidate_uuid: "e4f0b47924a5472db1668d8e66c776b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4d5d9af0a442229fad7bd504d94484"
I20260807 18:20:57.480093 31385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add7c11b40c147f9858570e2f130c417" candidate_uuid: "e4f0b47924a5472db1668d8e66c776b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9"
I20260807 18:20:57.480279 31784 raft_consensus.cc:3037] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:57.480396 31385 raft_consensus.cc:3037] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:57.483608 31385 raft_consensus.cc:2454] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f0b47924a5472db1668d8e66c776b0 in term 1.
I20260807 18:20:57.484197 31491 leader_election.cc:302] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba9dc30a856a4a628cfe62d1f41aa5c9, e4f0b47924a5472db1668d8e66c776b0; no voters:
I20260807 18:20:57.484519 31784 raft_consensus.cc:2454] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f0b47924a5472db1668d8e66c776b0 in term 1.
I20260807 18:20:57.484683 31889 raft_consensus.cc:2781] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:57.485539 31889 raft_consensus.cc:686] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [term 1 LEADER]: Becoming Leader. State: Replica: e4f0b47924a5472db1668d8e66c776b0, State: Running, Role: LEADER
I20260807 18:20:57.486133 31889 consensus_queue.cc:237] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } }
I20260807 18:20:57.492236 31179 catalog_manager.cc:5495] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f0b47924a5472db1668d8e66c776b0 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "e4f0b47924a5472db1668d8e66c776b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4d5d9af0a442229fad7bd504d94484" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f0b47924a5472db1668d8e66c776b0" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba9dc30a856a4a628cfe62d1f41aa5c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44991 } health_report { overall_health: UNKNOWN } } }
W20260807 18:20:57.510238 31857 tablet.cc:2369] T add7c11b40c147f9858570e2f130c417 P aa4d5d9af0a442229fad7bd504d94484: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:20:57.523032 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.503848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.29.156.193:44991)
0807 18:20:57.504038 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.504042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.504712 (+ 670us) server_negotiation.cc:407] Connection header received
0807 18:20:57.504999 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.505004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.505086 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.505210 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.507179 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.508157 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.509574 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.510462 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.510591 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.515967 (+ 5376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.515985 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.516002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.516050 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.519768 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.520614 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.520623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.520625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.520773 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.521226 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.521231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.521234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.521639 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.522001 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.522326 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.522560 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:20:57.584036 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.565973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.29.156.193:44991)
0807 18:20:57.566423 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.566430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.566889 (+ 459us) server_negotiation.cc:407] Connection header received
0807 18:20:57.567252 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.567263 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.567337 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.567551 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.569795 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.570685 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.572290 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.572868 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.573038 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.577945 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.577951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.577955 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.577994 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.581221 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.582004 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.582008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.582011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.582153 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.582653 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.582657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.582659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.582965 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.583189 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.583482 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.583767 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
W20260807 18:20:57.642566 31454 tablet.cc:2369] T add7c11b40c147f9858570e2f130c417 P ba9dc30a856a4a628cfe62d1f41aa5c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:57.644812 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.627313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.29.156.193:44991)
0807 18:20:57.627619 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.627627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.628023 (+ 396us) server_negotiation.cc:407] Connection header received
0807 18:20:57.628278 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.628283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.628356 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.628484 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.630558 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.631517 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.633080 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.633551 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.633704 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.638290 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.638296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.638299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.638350 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.641642 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.642456 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.642463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.642469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.642642 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.643082 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.643087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.643090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.643707 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.643971 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.644261 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.644456 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 18:20:57.646816 31330 tablet_service.cc:2908] Rejecting scan request for tablet add7c11b40c147f9858570e2f130c417: Uninitialized: clean time has not yet been initialized
I20260807 18:20:57.650595 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 31266
I20260807 18:20:57.665560 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.651462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.29.156.193:44991)
0807 18:20:57.651799 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.651806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.652191 (+ 385us) server_negotiation.cc:407] Connection header received
0807 18:20:57.652324 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.652327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.652406 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.652499 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.653727 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.654409 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.655765 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.656364 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.656488 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.659211 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.659216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.659219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.659254 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.662572 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.663369 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.663373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.663376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.663511 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.664126 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.664130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.664132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.664452 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.664696 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.665039 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.665228 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260807 18:20:57.666343 31444 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 18:20:57.678102 31659 tablet.cc:2369] T add7c11b40c147f9858570e2f130c417 P e4f0b47924a5472db1668d8e66c776b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:57.762990 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 31266
I20260807 18:20:57.787611 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 31460
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:20:57.804958 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.788765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.29.156.194:42329)
0807 18:20:57.788956 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.788960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.788979 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:20:57.789041 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.789046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.789116 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.789207 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.790519 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.791355 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.793101 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.793557 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.793665 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.798425 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.798444 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.798461 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.798551 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.801882 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.802676 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.802686 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.802688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.802932 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.803351 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.803355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.803357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.803870 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.804162 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.804454 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.804693 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:20:57.805732 31652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:57.900955 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.900386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:44991 (local address 127.29.156.194:52439)
0807 18:20:57.900640 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:57.900795 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:44991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 18:20:57.900972 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 31460
I20260807 18:20:57.932049 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 31667
Aug 07 18:20:57 dist-test-slave-sb2z krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:20:57.947801 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:57.932871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.29.156.195:37287)
0807 18:20:57.933056 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:57.933062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:57.933103 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:20:57.933203 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:57.933207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:57.933271 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:57.933358 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:57.934761 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.935480 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.936848 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:57.937229 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:57.937328 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:57.941737 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:57.941754 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:57.941777 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:57.941827 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:57.944878 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.945625 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.945632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.945635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.945768 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:57.946106 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:57.946109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:57.946111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:57.946479 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:57.946723 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:57.947202 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 18:20:57.947372 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:20:57.948499 31851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:58.040845 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 31667
I20260807 18:20:58.063285 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 31129
I20260807 18:20:58.064635 31219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:58.177644 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 31129
2026-08-07T18:20:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4713 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:58 dist-test-slave-sb2z krb5kdc[31921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:58 dist-test-slave-sb2z krb5kdc[31921](info): set up 2 sockets
Aug 07 18:20:58 dist-test-slave-sb2z krb5kdc[31921](info): commencing operation
krb5kdc: starting...
W20260807 18:20:59.463428 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:20:59 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:20:59.696664 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36521
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:00.037670 31944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:00.038069 31944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:00.038233 31944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:00.045816 31944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:00.045989 31944 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:21:00.046088 31944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:00.046214 31944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:00.046305 31944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:00.046401 31944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:00.057298 31944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:36521
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:00.058756 31944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:00.060302 31944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:00.074180 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:00.074162 31955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:00.074368 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:00.074955 31944 server_base.cc:1034] running on GCE node
I20260807 18:21:00.075940 31944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:00.077466 31944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:00.078754 31944 hybrid_clock.cc:648] HybridClock initialized: now 1786126860078661 us; error 75 us; skew 500 ppm
I20260807 18:21:00.079133 31944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:00 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:00.082616 31944 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:00.084434 31944 webserver.cc:460] Webserver started at http://127.29.156.254:34171/ using document root <none> and password file <none>
I20260807 18:21:00.085212 31944 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:00.085353 31944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:00.085711 31944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:00.088912 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "f0a687cdd5b8445f8df670837befc170"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "2fdcaa44d38eef5bf5e428060a57c6c9"
server_key_iv: "25e14129518174bd2f6fd98fcf12d18e"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.089783 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "f0a687cdd5b8445f8df670837befc170"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "2fdcaa44d38eef5bf5e428060a57c6c9"
server_key_iv: "25e14129518174bd2f6fd98fcf12d18e"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.095458 31944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:21:00.099303 31963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:00.100929 31944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:00.101178 31944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "f0a687cdd5b8445f8df670837befc170"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "2fdcaa44d38eef5bf5e428060a57c6c9"
server_key_iv: "25e14129518174bd2f6fd98fcf12d18e"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.101423 31944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:00.114379 31944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:00.118835 31944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:00.119151 31944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:00.143615 31944 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:36521
I20260807 18:21:00.143705 32058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:36521 every 8 connection(s)
I20260807 18:21:00.145104 31944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:00.149168 32059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:00.149577 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 31944
I20260807 18:21:00.149829 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:00.150579 30323 external_mini_cluster.cc:1428] Setting key 05f6806ef9a4c571dfce022c207dece3
I20260807 18:21:00.164695 32059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Bootstrap starting.
Aug 07 18:21:00 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:00.168232 32059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:00.169742 32059 log.cc:824] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:00.173516 32059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: No bootstrap required, opened a new log
I20260807 18:21:00.177704 32062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:00.153084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.29.156.254:36521)
0807 18:21:00.154037 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:00.154054 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:00.154117 (+ 63us) server_negotiation.cc:407] Connection header received
0807 18:21:00.156447 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:00.156481 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:00.157074 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:00.157595 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:00.159038 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.161188 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.162605 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.163734 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.163930 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:00.168255 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:00.168269 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:00.168290 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:00.168350 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:00.172706 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.173475 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.173486 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.173495 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.173745 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.174200 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.174205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.174208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.174955 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:00.175257 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:00.175696 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 18:21:00.176182 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":427,"threads_started":1}
I20260807 18:21:00.183128 32059 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } }
I20260807 18:21:00.183590 32059 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:00.183717 32059 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a687cdd5b8445f8df670837befc170, State: Initialized, Role: FOLLOWER
I20260807 18:21:00.184377 32059 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } }
I20260807 18:21:00.184712 32059 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:00.184861 32059 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:00.185042 32059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:00.187325 32059 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } }
I20260807 18:21:00.188004 32059 leader_election.cc:302] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0a687cdd5b8445f8df670837befc170; no voters:
I20260807 18:21:00.188637 32059 leader_election.cc:288] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:00.188802 32065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:00.189579 32065 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [term 1 LEADER]: Becoming Leader. State: Replica: f0a687cdd5b8445f8df670837befc170, State: Running, Role: LEADER
I20260807 18:21:00.190145 32065 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } }
I20260807 18:21:00.190838 32059 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:00.193243 32068 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0a687cdd5b8445f8df670837befc170. Latest consensus state: current_term: 1 leader_uuid: "f0a687cdd5b8445f8df670837befc170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } } }
I20260807 18:21:00.193727 32068 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:00.193981 32066 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0a687cdd5b8445f8df670837befc170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a687cdd5b8445f8df670837befc170" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36521 } } }
I20260807 18:21:00.194276 32066 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:00.196421 32073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:00.201900 32073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:00.212289 32073 catalog_manager.cc:1347] Generated new cluster ID: 02bb43b91ec84857a61f8b23b11d06ba
I20260807 18:21:00.212461 32073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:00.273001 32073 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:00.275031 32073 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:00.282618 32073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Generated new TSK 0
I20260807 18:21:00.283651 32073 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:00.351248 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36521
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:00.705888 32098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:00.706192 32098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:00.706377 32098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:00.713868 32098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:00.713987 32098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:00.714185 32098 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:00.714316 32098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:00.724659 32098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:00.726055 32098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:00.727481 32098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:00.741707 32108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:00.741786 32109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:00.741750 32112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:00.742913 32098 server_base.cc:1034] running on GCE node
I20260807 18:21:00.743690 32098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:00.744717 32098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:00.745985 32098 hybrid_clock.cc:648] HybridClock initialized: now 1786126860745945 us; error 76 us; skew 500 ppm
I20260807 18:21:00.746316 32098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:00 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:00.754294 32098 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:00.756232 32098 webserver.cc:460] Webserver started at http://127.29.156.193:33079/ using document root <none> and password file <none>
I20260807 18:21:00.757068 32098 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:00.757225 32098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:00.757606 32098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:00.761042 32098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "fceffd69017a4353944d63730b8c7cde"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "cbd926c3a46aa44298b95681f3ca44d3"
server_key_iv: "0bfa326fac0118912057aab7b7f91b7b"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.761950 32098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "fceffd69017a4353944d63730b8c7cde"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "cbd926c3a46aa44298b95681f3ca44d3"
server_key_iv: "0bfa326fac0118912057aab7b7f91b7b"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.768049 32098 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:21:00.771895 32120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:00.773306 32098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:00.773497 32098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "fceffd69017a4353944d63730b8c7cde"
format_stamp: "Formatted at 2026-08-07 18:21:00 on dist-test-slave-sb2z"
server_key: "cbd926c3a46aa44298b95681f3ca44d3"
server_key_iv: "0bfa326fac0118912057aab7b7f91b7b"
server_key_version: "encryptionkey@0"
I20260807 18:21:00.773751 32098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:00.788898 32098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:00.793720 32098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:00.794029 32098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:00.797214 32098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:00.797431 32098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:21:00.797605 32098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:00.797701 32098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:00 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:00.825834 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:00.797552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38069 (local address 127.29.156.254:36521)
0807 18:21:00.798082 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:00.798091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:00.799650 (+ 1559us) server_negotiation.cc:407] Connection header received
0807 18:21:00.802759 (+ 3109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:00.802767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:00.802870 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:00.803005 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:00.805798 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.806911 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.810419 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.811208 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.811364 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:00.817499 (+ 6135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:00.817509 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:00.817517 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:00.817593 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:00.821653 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.822907 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.822914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.822920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.823088 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.823801 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.823805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.823807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.824091 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:00.824208 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:00.825379 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0807 18:21:00.825570 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":253,"threads_started":1}
I20260807 18:21:00.828114 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:00.798304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.193:38069)
0807 18:21:00.799417 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:00.799491 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:00.802361 (+ 2870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:00.803333 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:00.803348 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:00.804394 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:00.805505 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:00.805557 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.807112 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.807120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:00.810177 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:00.810213 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.811733 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.811739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:00.812070 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:00.813000 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:00.813051 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:00.817128 (+ 4077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:00.821863 (+ 4735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:00.821877 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:00.821898 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:00.822723 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:00.823221 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:00.823252 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:00.823257 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:00.823623 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:00.824252 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:00.824265 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:00.824890 (+ 625us) client_negotiation.cc:769] Sending connection context
0807 18:21:00.825287 (+ 397us) client_negotiation.cc:240] Negotiation successful
0807 18:21:00.826045 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":384,"threads_started":1}
I20260807 18:21:00.851063 32098 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:42809
I20260807 18:21:00.851140 32292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:42809 every 8 connection(s)
I20260807 18:21:00.852892 32098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:00.855250 32294 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36521
I20260807 18:21:00.855598 32294 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:00.856019 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:00.838250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38215 (local address 127.29.156.254:36521)
0807 18:21:00.838487 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:00.838496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:00.839075 (+ 579us) server_negotiation.cc:407] Connection header received
0807 18:21:00.839242 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:00.839250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:00.839358 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:00.839504 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:00.840773 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.842216 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.844819 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.845271 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.845385 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:00.848713 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:00.848719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:00.848724 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:00.848765 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:00.852654 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.853581 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.853587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.853590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.853773 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:00.854347 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:00.854353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:00.854356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:00.854682 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:00.854880 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:00.855349 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 18:21:00.855624 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:21:00.856129 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:00.838242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.193:38215)
0807 18:21:00.838971 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:00.838988 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:00.839116 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:00.839668 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:00.839678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:00.840244 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:00.840572 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:00.840583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.842445 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.842464 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:00.844622 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:00.844638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:00.845524 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:00.845534 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:00.845832 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:00.846714 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:00.846748 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:00.848399 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:00.852811 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:00.852815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:00.852824 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:00.853403 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:00.853967 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:00.853975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:00.853979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:00.854164 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:00.854823 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:00.854828 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:00.855069 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:21:00.855224 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:21:00.855639 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":393,"threads_started":1}
I20260807 18:21:00.856101 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 32098
I20260807 18:21:00.856348 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:00.856705 32294 heartbeater.cc:511] Master 127.29.156.254:36521 requested a full tablet report, sending...
I20260807 18:21:00.857275 30323 external_mini_cluster.cc:1428] Setting key e1f30ce98e408e68b2937cabd9e06ef9
I20260807 18:21:00.859480 31999 ts_manager.cc:194] Registered new tserver with Master: fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809)
I20260807 18:21:00.861192 31999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:38069
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:00.911099 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36521
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:01.258174 32298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:01.258504 32298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:01.258693 32298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:01.266217 32298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:01.266353 32298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:01.266507 32298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:01.266644 32298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:01.277447 32298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:01.279022 32298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:01.280488 32298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:01.295363 32305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:01.295298 32310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:01.295383 32304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:01.296375 32298 server_base.cc:1034] running on GCE node
I20260807 18:21:01.297196 32298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:01.298287 32298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:01.299618 32298 hybrid_clock.cc:648] HybridClock initialized: now 1786126861299490 us; error 43 us; skew 500 ppm
I20260807 18:21:01.299968 32298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:01 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126861, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:01.303829 32298 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:01.305759 32298 webserver.cc:460] Webserver started at http://127.29.156.194:32913/ using document root <none> and password file <none>
I20260807 18:21:01.306627 32298 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:01.306775 32298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:01.307145 32298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:01.310376 32298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "d009c5dea31a40649e3aed75831f17a8"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "f700e71d83397b34d520dff49656e543"
server_key_iv: "c187bb7caa5b4a6e80e0a7aae8020801"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.311261 32298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "d009c5dea31a40649e3aed75831f17a8"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "f700e71d83397b34d520dff49656e543"
server_key_iv: "c187bb7caa5b4a6e80e0a7aae8020801"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.317132 32298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:21:01.322113 32320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:01.324282 32298 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:21:01.324487 32298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "d009c5dea31a40649e3aed75831f17a8"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "f700e71d83397b34d520dff49656e543"
server_key_iv: "c187bb7caa5b4a6e80e0a7aae8020801"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.324780 32298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:01.353382 32298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:01.358613 32298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:01.358973 32298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:01.361974 32298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:01.362115 32298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:01.362284 32298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:01.362365 32298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:01 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126861, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:01.390339 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.362939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35973 (local address 127.29.156.254:36521)
0807 18:21:01.363422 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:01.363428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:01.365056 (+ 1628us) server_negotiation.cc:407] Connection header received
0807 18:21:01.368875 (+ 3819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:01.368883 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:01.368999 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:01.369147 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:01.371666 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.373224 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.375860 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.376481 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.376641 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.383071 (+ 6430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:01.383076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:01.383080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:01.383118 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:01.386443 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.387513 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.387553 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.387561 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.387771 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.388349 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.388355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.388358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.388672 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:01.388845 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:01.389661 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 18:21:01.389921 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260807 18:21:01.391479 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.363766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.194:35973)
0807 18:21:01.364807 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:01.364875 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:01.368414 (+ 3539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:01.369511 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:01.369526 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:01.370469 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:01.371361 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.371418 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.373494 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.373506 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.375666 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.375681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.376779 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.376788 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.377144 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.378110 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:01.378193 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:01.382697 (+ 4504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:01.386611 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.386625 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.386650 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.387328 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.387924 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.387937 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.387941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.388184 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.388817 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:01.388826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:01.389266 (+ 440us) client_negotiation.cc:769] Sending connection context
0807 18:21:01.389554 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 18:21:01.390155 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":404,"threads_started":1}
I20260807 18:21:01.420094 32298 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:42635
I20260807 18:21:01.420190 32478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:42635 every 8 connection(s)
I20260807 18:21:01.421072 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.400160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.194:59531)
0807 18:21:01.404132 (+ 3972us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:01.404155 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:01.404308 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:01.404791 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:01.404799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:01.405669 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:01.406202 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.406223 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.407597 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.407601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.408744 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.408756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.410172 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.410180 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.410372 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.411148 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:01.411188 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:01.412202 (+ 1014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:01.417284 (+ 5082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.417290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.417295 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.417871 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.418533 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.418542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.418548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.418774 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.419867 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:01.419875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:01.420182 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 18:21:01.420364 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 18:21:01.420690 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3814,"thread_start_us":3672,"threads_started":1}
I20260807 18:21:01.421655 32298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:01.421931 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.400113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:59531 (local address 127.29.156.254:36521)
0807 18:21:01.400375 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:01.400382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:01.404284 (+ 3902us) server_negotiation.cc:407] Connection header received
0807 18:21:01.404455 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:01.404465 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:01.404575 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:01.404718 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:01.406428 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.407390 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.409038 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.409897 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.410086 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.412427 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:01.412438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:01.412445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:01.412497 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:01.417099 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.418100 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.418109 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.418116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.418328 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.418930 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.418937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.418941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.419578 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:01.419778 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:01.421442 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0807 18:21:01.421636 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:21:01.424047 32479 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36521
I20260807 18:21:01.424319 32479 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:01.425381 32479 heartbeater.cc:511] Master 127.29.156.254:36521 requested a full tablet report, sending...
I20260807 18:21:01.426215 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 32298
I20260807 18:21:01.426486 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:01.426993 31999 ts_manager.cc:194] Registered new tserver with Master: d009c5dea31a40649e3aed75831f17a8 (127.29.156.194:42635)
I20260807 18:21:01.427070 30323 external_mini_cluster.cc:1428] Setting key dd2acd37a913511eff0af5debc7ccf69
I20260807 18:21:01.428138 31999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:35973
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:01.480269 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36521
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:01.830530 32484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:01.830871 32484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:01.831063 32484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:01.838891 32484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:01.839064 32484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:01.839217 32484 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:01.839356 32484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:01.849704 32484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:01.851209 32484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:01.852679 32484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:21:01.864616 32294 heartbeater.cc:503] Master 127.29.156.254:36521 was elected leader, sending a full tablet report...
W20260807 18:21:01.869853 32489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:01.869886 32491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:01.871876 32495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:01.877677 32484 server_base.cc:1034] running on GCE node
I20260807 18:21:01.878743 32484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:01.880084 32484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:01.881357 32484 hybrid_clock.cc:648] HybridClock initialized: now 1786126861881260 us; error 67 us; skew 500 ppm
I20260807 18:21:01.881738 32484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:01 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126861, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:01.885764 32484 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:01.887734 32484 webserver.cc:460] Webserver started at http://127.29.156.195:42561/ using document root <none> and password file <none>
I20260807 18:21:01.888583 32484 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:01.888737 32484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:01.889108 32484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:01.892797 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "474091b269de4442b4676075c8228dc2"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "5eb7d14167ab14c4bb87676d117f5e71"
server_key_iv: "c27b36ffd6fadc41ff9145b46f8cd11b"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.893790 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "474091b269de4442b4676075c8228dc2"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "5eb7d14167ab14c4bb87676d117f5e71"
server_key_iv: "c27b36ffd6fadc41ff9145b46f8cd11b"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.900430 32484 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:21:01.904670 32503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:01.906334 32484 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:21:01.906603 32484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "474091b269de4442b4676075c8228dc2"
format_stamp: "Formatted at 2026-08-07 18:21:01 on dist-test-slave-sb2z"
server_key: "5eb7d14167ab14c4bb87676d117f5e71"
server_key_iv: "c27b36ffd6fadc41ff9145b46f8cd11b"
server_key_version: "encryptionkey@0"
I20260807 18:21:01.906847 32484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:01.937881 32484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:01.943184 32484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:01.943508 32484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:01.947256 32484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:01.947420 32484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:01.947624 32484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:01.947731 32484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:01 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126861, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:01.994432 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.948431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.195:35509)
0807 18:21:01.959093 (+ 10662us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:01.959180 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:01.963833 (+ 4653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:01.965237 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:01.965254 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:01.966410 (+ 1156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:01.967624 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.967667 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.969957 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.969966 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.973208 (+ 3242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:01.973226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.975053 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.975068 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:01.975500 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.976622 (+ 1122us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:01.976667 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:01.982055 (+ 5388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:01.987397 (+ 5342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.987408 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.987429 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.988368 (+ 939us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.988984 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:01.988989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:01.988992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:01.989350 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:01.990270 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:01.990280 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:01.990963 (+ 683us) client_negotiation.cc:769] Sending connection context
0807 18:21:01.991375 (+ 412us) client_negotiation.cc:240] Negotiation successful
0807 18:21:01.992197 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10193,"thread_start_us":9897,"threads_started":1}
I20260807 18:21:01.995872 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:01.947779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:35509 (local address 127.29.156.254:36521)
0807 18:21:01.960324 (+ 12545us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:01.960333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:01.960369 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:21:01.964337 (+ 3968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:01.964348 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:01.964481 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:01.964615 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:01.967950 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.969634 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.973456 (+ 3822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:01.974383 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:01.974523 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:01.982477 (+ 7954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:01.982488 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:01.982495 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:01.982571 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:01.987153 (+ 4582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.988597 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.988606 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.988612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.988807 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:01.989522 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:01.989541 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:01.989547 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:01.990031 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:01.990188 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:01.994656 (+ 4468us) server_negotiation.cc:299] Negotiation successful
0807 18:21:01.995059 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12360,"thread_start_us":12187,"threads_started":1}
I20260807 18:21:02.024580 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.002968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:55967 (local address 127.29.156.254:36521)
0807 18:21:02.003183 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.003191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.003946 (+ 755us) server_negotiation.cc:407] Connection header received
0807 18:21:02.004380 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.004387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.004502 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.004632 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:02.006916 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.008353 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.011094 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.011946 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.012079 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.015743 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:02.015753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:02.015760 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:02.015814 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:02.020323 (+ 4509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.021587 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.021593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.021596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.021762 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.022268 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.022276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.022282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.022763 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:02.022941 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.023764 (+ 823us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.024095 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:21:02.024973 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.003070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.29.156.195:55967)
0807 18:21:02.003804 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.003835 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.004086 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.004926 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.004933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.005908 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:02.006698 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.006721 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.008551 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.008560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.010861 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.010882 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.012582 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.012593 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.012839 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.013779 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:02.013806 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:02.015426 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:02.020529 (+ 5103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:02.020540 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:02.020548 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:02.021370 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:02.021924 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:02.021931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:02.021936 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:02.022105 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:02.022955 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:02.022963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:02.023374 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.023627 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.024161 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":402,"threads_started":1}
I20260807 18:21:02.043329 32484 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:35799
I20260807 18:21:02.043372 32673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:35799 every 8 connection(s)
I20260807 18:21:02.044796 32484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:02.047096 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 32484
I20260807 18:21:02.047250 32675 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36521
I20260807 18:21:02.047326 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:02.047443 32675 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:02.048033 30323 external_mini_cluster.cc:1428] Setting key 749dfb6b4d813eee91ad4d473b55745b
I20260807 18:21:02.048242 32675 heartbeater.cc:511] Master 127.29.156.254:36521 requested a full tablet report, sending...
I20260807 18:21:02.049923 31996 ts_manager.cc:194] Registered new tserver with Master: 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.051433 31996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:35509
I20260807 18:21:02.053988 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:21:02 dist-test-slave-sb2z krb5kdc[31921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:21:02 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:02.122280 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.105580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.29.156.254:36521)
0807 18:21:02.105829 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.105838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.106298 (+ 460us) server_negotiation.cc:407] Connection header received
0807 18:21:02.106534 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.106543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.106687 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.106840 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:02.108533 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.109864 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.111367 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.112079 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.112209 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.116656 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:02.116663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:02.116667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:02.116710 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:02.119485 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.120250 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.120254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.120257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.120432 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.120873 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.120887 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.120889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.121218 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:02.121509 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.121716 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.121886 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:21:02.126878 31996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:02.135030 31996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:02.178824 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.150872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:42809 (local address 127.0.0.1:55468)
0807 18:21:02.160597 (+ 9725us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.160677 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.161018 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.167129 (+ 6111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.167137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.167174 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.168556 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.168568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.170999 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.171005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.174506 (+ 3501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.174522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.177374 (+ 2852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.177383 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.177761 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.177848 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.178024 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.178312 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9273,"mutex_wait_us":628,"thread_start_us":14405,"threads_started":1}
I20260807 18:21:02.180135 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.151139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.29.156.193:42809)
0807 18:21:02.166161 (+ 15022us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.166169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.166196 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:21:02.166278 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.166284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.166555 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.166780 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.168772 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.170800 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.174717 (+ 3917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.176772 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.176961 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.177150 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.179301 (+ 2151us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.179568 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14820,"thread_start_us":14614,"threads_started":1}
I20260807 18:21:02.181389 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.150997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:35799 (local address 127.0.0.1:39712)
0807 18:21:02.165563 (+ 14566us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.165589 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.165840 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.167199 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.167206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.167230 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.168062 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.168078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.172961 (+ 4883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.172970 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.176948 (+ 3978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.176973 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.180393 (+ 3420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.180402 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.180606 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.180645 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.180733 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.180895 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14391,"thread_start_us":8931,"threads_started":1}
I20260807 18:21:02.181591 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.151770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.29.156.195:35799)
0807 18:21:02.162040 (+ 10270us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.162054 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.166038 (+ 3984us) server_negotiation.cc:407] Connection header received
0807 18:21:02.166193 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.166202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.166556 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.166780 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.168433 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.172747 (+ 4314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.178466 (+ 5719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.180122 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.180423 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.180616 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.180825 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.181132 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9901,"thread_start_us":11895,"threads_started":1}
I20260807 18:21:02.184877 32193 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 (DEFAULT_TABLE table=test-table [id=778c01c8723745fa98835df6dce6470e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.185906 32579 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 (DEFAULT_TABLE table=test-table [id=778c01c8723745fa98835df6dce6470e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.186578 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0a3f844334f2d81364b5ec998e661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.187500 32579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0a3f844334f2d81364b5ec998e661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.188480 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.162958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.29.156.194:42635)
0807 18:21:02.165233 (+ 2275us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.165258 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.178325 (+ 13067us) server_negotiation.cc:407] Connection header received
0807 18:21:02.178723 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.178733 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.179065 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.179270 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.180594 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.182359 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.185403 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.186999 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.187213 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.187404 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.187898 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.188154 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2093,"thread_start_us":1947,"threads_started":1}
I20260807 18:21:02.188593 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.162672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42635 (local address 127.0.0.1:44054)
0807 18:21:02.178187 (+ 15515us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.178208 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.178476 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.179601 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.179605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.179632 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.180326 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.180343 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.182576 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.182583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.185187 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.185203 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.187336 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.187344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.187661 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.187698 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.187805 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.187978 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15315,"thread_start_us":15110,"threads_started":1}
I20260807 18:21:02.195127 32391 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 (DEFAULT_TABLE table=test-table [id=778c01c8723745fa98835df6dce6470e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.196676 32391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0a3f844334f2d81364b5ec998e661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.201179 32707 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Bootstrap starting.
I20260807 18:21:02.204002 32708 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Bootstrap starting.
I20260807 18:21:02.205381 32707 tablet_bootstrap.cc:654] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.206843 32707 log.cc:824] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:02.208099 32708 tablet_bootstrap.cc:654] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.209309 32708 log.cc:824] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:02.211301 32707 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: No bootstrap required, opened a new log
I20260807 18:21:02.211628 32707 ts_tablet_manager.cc:1397] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:21:02.212131 32713 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Bootstrap starting.
I20260807 18:21:02.212940 32708 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: No bootstrap required, opened a new log
I20260807 18:21:02.213343 32708 ts_tablet_manager.cc:1397] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 18:21:02.215155 32713 tablet_bootstrap.cc:654] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.216241 32713 log.cc:824] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:02.218859 32713 tablet_bootstrap.cc:492] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: No bootstrap required, opened a new log
I20260807 18:21:02.219192 32713 ts_tablet_manager.cc:1397] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260807 18:21:02.225250 32707 raft_consensus.cc:348] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.225971 32707 raft_consensus.cc:374] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.225791 32708 raft_consensus.cc:348] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.226184 32707 raft_consensus.cc:729] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceffd69017a4353944d63730b8c7cde, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.226326 32708 raft_consensus.cc:374] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.226459 32708 raft_consensus.cc:729] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474091b269de4442b4676075c8228dc2, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.227084 32708 consensus_queue.cc:260] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.227046 32707 consensus_queue.cc:260] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.227805 32713 raft_consensus.cc:348] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.228458 32713 raft_consensus.cc:374] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.228565 32675 heartbeater.cc:503] Master 127.29.156.254:36521 was elected leader, sending a full tablet report...
I20260807 18:21:02.228668 32713 raft_consensus.cc:729] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d009c5dea31a40649e3aed75831f17a8, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.229468 32713 consensus_queue.cc:260] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.229998 32707 ts_tablet_manager.cc:1428] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260807 18:21:02.231703 32708 ts_tablet_manager.cc:1428] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Time spent starting tablet: real 0.018s user 0.009s sys 0.007s
I20260807 18:21:02.234056 32479 heartbeater.cc:503] Master 127.29.156.254:36521 was elected leader, sending a full tablet report...
I20260807 18:21:02.234896 32713 ts_tablet_manager.cc:1428] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Time spent starting tablet: real 0.016s user 0.003s sys 0.010s
W20260807 18:21:02.296650 32676 tablet.cc:2369] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:21:02.355998 32295 tablet.cc:2369] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:02.360993 32717 raft_consensus.cc:482] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:02.361663 32717 raft_consensus.cc:504] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.363802 32717 leader_election.cc:288] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.373597 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.363985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:42809 (local address 127.29.156.194:50627)
0807 18:21:02.365811 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.365836 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.366128 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.366599 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.366603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.366627 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.367259 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.367275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.368775 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.368781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.370927 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.370940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.372607 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.372616 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.372843 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.372884 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.372981 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.373131 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"mutex_wait_us":24,"spinlock_wait_cycles":25344,"thread_start_us":1247,"threads_started":1}
I20260807 18:21:02.373624 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.364071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:50627 (local address 127.29.156.193:42809)
0807 18:21:02.364373 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.364380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.366055 (+ 1675us) server_negotiation.cc:407] Connection header received
0807 18:21:02.366318 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.366325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.366440 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.366560 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.367446 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.368630 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.371166 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.372279 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.372416 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.372474 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.373146 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.373320 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:21:02.374094 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.364373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:60981 (local address 127.29.156.195:35799)
0807 18:21:02.364723 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.364734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.366758 (+ 2024us) server_negotiation.cc:407] Connection header received
0807 18:21:02.366948 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.366954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.367032 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.367158 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.368230 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.369274 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.371456 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.372753 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.372930 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.373014 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.373657 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.373836 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:21:02.374370 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.365479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:35799 (local address 127.29.156.194:60981)
0807 18:21:02.366613 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.366636 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.366789 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.367214 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.367218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.367243 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.367997 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.368012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.369464 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.369468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.371283 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.371299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.373234 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.373241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.373446 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.373481 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.373605 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.373764 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":45,"thread_start_us":600,"threads_started":1}
I20260807 18:21:02.375002 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0a3f844334f2d81364b5ec998e661" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceffd69017a4353944d63730b8c7cde" is_pre_election: true
I20260807 18:21:02.375696 32226 raft_consensus.cc:2454] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 0.
I20260807 18:21:02.375512 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0a3f844334f2d81364b5ec998e661" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2" is_pre_election: true
I20260807 18:21:02.376227 32615 raft_consensus.cc:2454] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 0.
I20260807 18:21:02.376497 32326 leader_election.cc:302] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d009c5dea31a40649e3aed75831f17a8, fceffd69017a4353944d63730b8c7cde; no voters:
I20260807 18:21:02.377229 32717 raft_consensus.cc:2781] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:02.377434 32717 raft_consensus.cc:482] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:02.377604 32717 raft_consensus.cc:3037] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.380195 32717 raft_consensus.cc:504] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.380870 32717 leader_election.cc:288] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 election: Requested vote from peers fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.381539 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0a3f844334f2d81364b5ec998e661" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2"
I20260807 18:21:02.381637 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0a3f844334f2d81364b5ec998e661" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceffd69017a4353944d63730b8c7cde"
I20260807 18:21:02.381778 32615 raft_consensus.cc:3037] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.381909 32226 raft_consensus.cc:3037] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.385308 32226 raft_consensus.cc:2454] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 1.
I20260807 18:21:02.385660 32615 raft_consensus.cc:2454] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 1.
I20260807 18:21:02.385821 32326 leader_election.cc:302] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d009c5dea31a40649e3aed75831f17a8, fceffd69017a4353944d63730b8c7cde; no voters:
I20260807 18:21:02.386363 32717 raft_consensus.cc:2781] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:02.386943 32717 raft_consensus.cc:686] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 1 LEADER]: Becoming Leader. State: Replica: d009c5dea31a40649e3aed75831f17a8, State: Running, Role: LEADER
I20260807 18:21:02.387458 32717 consensus_queue.cc:237] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } }
I20260807 18:21:02.393836 31998 catalog_manager.cc:5495] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d009c5dea31a40649e3aed75831f17a8 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } health_report { overall_health: HEALTHY } } }
W20260807 18:21:02.424527 32480 tablet.cc:2369] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:02.430742 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.422785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.29.156.194:42635)
0807 18:21:02.423044 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.423053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.423246 (+ 193us) server_negotiation.cc:407] Connection header received
0807 18:21:02.423661 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.423665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.423756 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.423888 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:02.424472 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.425741 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.427928 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.428527 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.428697 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.429346 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.430035 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.430170 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.430307 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.430382 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:21:02.440145 32615 raft_consensus.cc:1261] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Refusing update from remote peer d009c5dea31a40649e3aed75831f17a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.440177 32226 raft_consensus.cc:1261] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Refusing update from remote peer d009c5dea31a40649e3aed75831f17a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.441293 32722 consensus_queue.cc:1035] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.441799 32717 consensus_queue.cc:1035] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.456362 32728 mvcc.cc:205] Tried to move back new op lower bound from 7315975628544405504 to 7315975628344381440. Current Snapshot: MvccSnapshot[applied={T|T < 7315975628544405504}]
I20260807 18:21:02.476985 31999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:02.477452 31999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:02.481779 31999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Sending DeleteTablet for 3 replicas of tablet 3ea0a3f844334f2d81364b5ec998e661
I20260807 18:21:02.483294 32193 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:55468
I20260807 18:21:02.483768 32579 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39712
I20260807 18:21:02.483731 32391 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea0a3f844334f2d81364b5ec998e661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:44054
I20260807 18:21:02.489610 31999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:02.490765 31999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:02.497113 32740 tablet_replica.cc:330] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: stopping tablet replica
I20260807 18:21:02.497299 32742 tablet_replica.cc:330] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: stopping tablet replica
I20260807 18:21:02.497903 32740 raft_consensus.cc:2229] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:02.497958 32742 raft_consensus.cc:2229] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:02.498659 32740 raft_consensus.cc:2258] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:02.498926 32742 raft_consensus.cc:2258] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:02.499955 32741 tablet_replica.cc:330] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: stopping tablet replica
I20260807 18:21:02.500420 32741 raft_consensus.cc:2229] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:02.501017 32741 raft_consensus.cc:2258] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:02.503293 32742 ts_tablet_manager.cc:1905] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:02.504431 32740 ts_tablet_manager.cc:1905] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:02.506079 32193 tablet_service.cc:1467] Processing CreateTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 (DEFAULT_TABLE table=test-table [id=4b830c0c59894238b115ca0e5f386381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.507412 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07420d6aa87a4c9a9b13d71b60191801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.507427 32391 tablet_service.cc:1467] Processing CreateTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 (DEFAULT_TABLE table=test-table [id=4b830c0c59894238b115ca0e5f386381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.508077 32391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07420d6aa87a4c9a9b13d71b60191801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.509938 32579 tablet_service.cc:1467] Processing CreateTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 (DEFAULT_TABLE table=test-table [id=4b830c0c59894238b115ca0e5f386381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:02.510464 32579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07420d6aa87a4c9a9b13d71b60191801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.514935 32742 ts_tablet_manager.cc:1918] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:02.515090 32707 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Bootstrap starting.
I20260807 18:21:02.515194 32742 log.cc:1196] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/3ea0a3f844334f2d81364b5ec998e661
I20260807 18:21:02.516041 32741 ts_tablet_manager.cc:1905] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:02.517194 32707 tablet_bootstrap.cc:654] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.521636 32708 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Bootstrap starting.
I20260807 18:21:02.522500 32713 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Bootstrap starting.
I20260807 18:21:02.523870 32707 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: No bootstrap required, opened a new log
I20260807 18:21:02.524029 32708 tablet_bootstrap.cc:654] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.524115 32707 ts_tablet_manager.cc:1397] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20260807 18:21:02.525120 32713 tablet_bootstrap.cc:654] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.525314 32707 raft_consensus.cc:348] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.525599 32707 raft_consensus.cc:374] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.525754 32707 raft_consensus.cc:729] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceffd69017a4353944d63730b8c7cde, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.526082 32707 consensus_queue.cc:260] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.528752 32741 ts_tablet_manager.cc:1918] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:02.528995 32741 log.cc:1196] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/3ea0a3f844334f2d81364b5ec998e661
I20260807 18:21:02.529500 32740 ts_tablet_manager.cc:1918] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:02.529711 32740 log.cc:1196] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/3ea0a3f844334f2d81364b5ec998e661
I20260807 18:21:02.529827 32741 ts_tablet_manager.cc:1939] T 3ea0a3f844334f2d81364b5ec998e661 P 474091b269de4442b4676075c8228dc2: Deleting consensus metadata
I20260807 18:21:02.529736 32742 ts_tablet_manager.cc:1939] T 3ea0a3f844334f2d81364b5ec998e661 P d009c5dea31a40649e3aed75831f17a8: Deleting consensus metadata
I20260807 18:21:02.530453 32740 ts_tablet_manager.cc:1939] T 3ea0a3f844334f2d81364b5ec998e661 P fceffd69017a4353944d63730b8c7cde: Deleting consensus metadata
I20260807 18:21:02.532483 32707 ts_tablet_manager.cc:1428] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260807 18:21:02.534224 32713 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: No bootstrap required, opened a new log
I20260807 18:21:02.534425 32713 ts_tablet_manager.cc:1397] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260807 18:21:02.534452 31972 catalog_manager.cc:4841] TS fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809): tablet 3ea0a3f844334f2d81364b5ec998e661 (table test-table [id=778c01c8723745fa98835df6dce6470e]) successfully deleted
I20260807 18:21:02.534956 31971 catalog_manager.cc:4841] TS d009c5dea31a40649e3aed75831f17a8 (127.29.156.194:42635): tablet 3ea0a3f844334f2d81364b5ec998e661 (table test-table [id=778c01c8723745fa98835df6dce6470e]) successfully deleted
I20260807 18:21:02.535251 31970 catalog_manager.cc:4841] TS 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799): tablet 3ea0a3f844334f2d81364b5ec998e661 (table test-table [id=778c01c8723745fa98835df6dce6470e]) successfully deleted
I20260807 18:21:02.536808 32713 raft_consensus.cc:348] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.537122 32713 raft_consensus.cc:374] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.537175 32708 tablet_bootstrap.cc:492] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: No bootstrap required, opened a new log
I20260807 18:21:02.537282 32713 raft_consensus.cc:729] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d009c5dea31a40649e3aed75831f17a8, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.537384 32708 ts_tablet_manager.cc:1397] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.003s
I20260807 18:21:02.537626 32713 consensus_queue.cc:260] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.538312 32708 raft_consensus.cc:348] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.538546 32713 ts_tablet_manager.cc:1428] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 18:21:02.538591 32708 raft_consensus.cc:374] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.538719 32708 raft_consensus.cc:729] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474091b269de4442b4676075c8228dc2, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.539023 32708 consensus_queue.cc:260] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.540387 32708 ts_tablet_manager.cc:1428] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 18:21:02.555809 32715 raft_consensus.cc:482] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:02.556085 32715 raft_consensus.cc:504] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.558046 32715 leader_election.cc:288] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d009c5dea31a40649e3aed75831f17a8 (127.29.156.194:42635), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.567938 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.559512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:58071 (local address 127.29.156.195:35799)
0807 18:21:02.559767 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.559774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.560728 (+ 954us) server_negotiation.cc:407] Connection header received
0807 18:21:02.561114 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.561121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.561221 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.561345 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.562241 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.563343 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.565281 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.566346 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.566524 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.566672 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.567422 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.567593 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:21:02.567966 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.558891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:35799 (local address 127.29.156.193:58071)
0807 18:21:02.560591 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.560613 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.560861 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.561393 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.561397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.561428 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.561980 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.561997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.563473 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.563478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.565087 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.565100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.566892 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.566900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.567182 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.567227 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.567373 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.567517 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1571,"thread_start_us":860,"threads_started":1}
I20260807 18:21:02.569073 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07420d6aa87a4c9a9b13d71b60191801" candidate_uuid: "fceffd69017a4353944d63730b8c7cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2" is_pre_election: true
I20260807 18:21:02.569432 32615 raft_consensus.cc:2454] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceffd69017a4353944d63730b8c7cde in term 0.
I20260807 18:21:02.569636 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.558526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40751 (local address 127.29.156.194:42635)
0807 18:21:02.558790 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.558800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.560145 (+ 1345us) server_negotiation.cc:407] Connection header received
0807 18:21:02.560526 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.560538 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.560604 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.560714 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:02.561661 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.562970 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.566482 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.568296 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.568463 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.568563 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.569056 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.569190 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:21:02.569852 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.558435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42635 (local address 127.29.156.193:40751)
0807 18:21:02.559986 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.560022 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.560298 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.560730 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.560735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.560761 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:02.561469 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.561486 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.563139 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.563145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.566256 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.566274 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.568662 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.568668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.568867 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.568892 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.569023 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.569172 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":50,"thread_start_us":1781,"threads_started":1}
I20260807 18:21:02.570504 32122 leader_election.cc:302] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 474091b269de4442b4676075c8228dc2, fceffd69017a4353944d63730b8c7cde; no voters:
I20260807 18:21:02.571202 32715 raft_consensus.cc:2781] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:02.570994 32419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07420d6aa87a4c9a9b13d71b60191801" candidate_uuid: "fceffd69017a4353944d63730b8c7cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d009c5dea31a40649e3aed75831f17a8" is_pre_election: true
I20260807 18:21:02.571327 32715 raft_consensus.cc:482] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:02.571415 32715 raft_consensus.cc:3037] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.571571 32419 raft_consensus.cc:2454] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceffd69017a4353944d63730b8c7cde in term 0.
I20260807 18:21:02.574169 32715 raft_consensus.cc:504] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.574828 32715 leader_election.cc:288] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [CANDIDATE]: Term 1 election: Requested vote from peers d009c5dea31a40649e3aed75831f17a8 (127.29.156.194:42635), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.575395 32419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07420d6aa87a4c9a9b13d71b60191801" candidate_uuid: "fceffd69017a4353944d63730b8c7cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d009c5dea31a40649e3aed75831f17a8"
I20260807 18:21:02.575452 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07420d6aa87a4c9a9b13d71b60191801" candidate_uuid: "fceffd69017a4353944d63730b8c7cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2"
I20260807 18:21:02.575660 32419 raft_consensus.cc:3037] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.575749 32615 raft_consensus.cc:3037] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.578740 32419 raft_consensus.cc:2454] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceffd69017a4353944d63730b8c7cde in term 1.
I20260807 18:21:02.579217 32123 leader_election.cc:302] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d009c5dea31a40649e3aed75831f17a8, fceffd69017a4353944d63730b8c7cde; no voters:
I20260807 18:21:02.579461 32615 raft_consensus.cc:2454] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceffd69017a4353944d63730b8c7cde in term 1.
I20260807 18:21:02.579619 32715 raft_consensus.cc:2781] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:02.580331 32715 raft_consensus.cc:686] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 1 LEADER]: Becoming Leader. State: Replica: fceffd69017a4353944d63730b8c7cde, State: Running, Role: LEADER
I20260807 18:21:02.580776 32715 consensus_queue.cc:237] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.584395 31996 catalog_manager.cc:5495] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde reported cstate change: term changed from 0 to 1, leader changed from <none> to fceffd69017a4353944d63730b8c7cde (127.29.156.193). New cstate: current_term: 1 leader_uuid: "fceffd69017a4353944d63730b8c7cde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:21:02 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:02.642668 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.625909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36521 (local address 127.0.0.1:60740)
0807 18:21:02.626656 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.626681 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.626881 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.627342 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.627346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.628268 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:02.628575 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.628586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.629845 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.629853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.631065 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.631078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.632000 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.632006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.632203 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.632983 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:02.633001 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:02.635934 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:02.639832 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:02.639847 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:02.639857 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:02.640321 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:02.640781 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:02.640787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:02.640791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:02.640950 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:02.641480 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:02.641489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:02.641674 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.641847 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.642183 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":4,"thread_start_us":375,"threads_started":1}
I20260807 18:21:02.642810 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.626045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.29.156.254:36521)
0807 18:21:02.627078 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.627082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.627100 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:21:02.627139 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.627143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.627194 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.627314 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:02.628761 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.629715 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.631219 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.631798 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.631934 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.636216 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:02.636222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:02.636225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:02.636258 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:02.639680 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.640452 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.640459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.640462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.640665 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:02.641069 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:02.641074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:02.641079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:02.641347 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:02.641505 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.641927 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.642316 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":899,"thread_start_us":729,"threads_started":1}
I20260807 18:21:02.648571 31996 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:60740:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:21:02.649752 31996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:02.657099 32193 tablet_service.cc:1467] Processing CreateTablet for tablet f8517b9a6ede483e85e9b5bf3536b400 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=744cc1747e58497ead5ee797ee3dd27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:02.657305 32391 tablet_service.cc:1467] Processing CreateTablet for tablet f8517b9a6ede483e85e9b5bf3536b400 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=744cc1747e58497ead5ee797ee3dd27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:02.657733 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8517b9a6ede483e85e9b5bf3536b400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.658010 32391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8517b9a6ede483e85e9b5bf3536b400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.661109 32579 tablet_service.cc:1467] Processing CreateTablet for tablet f8517b9a6ede483e85e9b5bf3536b400 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=744cc1747e58497ead5ee797ee3dd27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:02.661782 32579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8517b9a6ede483e85e9b5bf3536b400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:02.666011 32707 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: Bootstrap starting.
I20260807 18:21:02.668062 32707 tablet_bootstrap.cc:654] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.669337 32713 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: Bootstrap starting.
I20260807 18:21:02.670667 32708 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: Bootstrap starting.
I20260807 18:21:02.671545 32713 tablet_bootstrap.cc:654] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.672719 32708 tablet_bootstrap.cc:654] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:02.674988 32713 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: No bootstrap required, opened a new log
I20260807 18:21:02.675195 32713 ts_tablet_manager.cc:1397] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:21:02.676229 32713 raft_consensus.cc:348] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.676568 32713 raft_consensus.cc:374] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.676666 32713 raft_consensus.cc:729] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d009c5dea31a40649e3aed75831f17a8, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.677052 32713 consensus_queue.cc:260] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.677541 32722 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.677930 32722 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.679198 32713 ts_tablet_manager.cc:1428] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 18:21:02.683233 32707 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: No bootstrap required, opened a new log
I20260807 18:21:02.683406 32707 ts_tablet_manager.cc:1397] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.003s
I20260807 18:21:02.683977 32708 tablet_bootstrap.cc:492] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: No bootstrap required, opened a new log
I20260807 18:21:02.684165 32708 ts_tablet_manager.cc:1397] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260807 18:21:02.684319 32707 raft_consensus.cc:348] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.684582 32707 raft_consensus.cc:374] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.684737 32707 raft_consensus.cc:729] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceffd69017a4353944d63730b8c7cde, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.685038 32707 consensus_queue.cc:260] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.685154 32708 raft_consensus.cc:348] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.685557 32708 raft_consensus.cc:374] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:02.685668 32708 raft_consensus.cc:729] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474091b269de4442b4676075c8228dc2, State: Initialized, Role: FOLLOWER
I20260807 18:21:02.685621 32715 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.685927 32715 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.685969 32708 consensus_queue.cc:260] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.686668 32708 ts_tablet_manager.cc:1428] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 18:21:02.686491 32716 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.686996 32716 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.688108 32707 ts_tablet_manager.cc:1428] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:21:02.779896 32722 raft_consensus.cc:482] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:02.780270 32722 raft_consensus.cc:504] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.781178 32722 leader_election.cc:288] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.781798 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8517b9a6ede483e85e9b5bf3536b400" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceffd69017a4353944d63730b8c7cde" is_pre_election: true
I20260807 18:21:02.781934 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8517b9a6ede483e85e9b5bf3536b400" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2" is_pre_election: true
I20260807 18:21:02.782120 32226 raft_consensus.cc:2454] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 0.
I20260807 18:21:02.782249 32615 raft_consensus.cc:2454] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 0.
I20260807 18:21:02.782718 32326 leader_election.cc:302] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d009c5dea31a40649e3aed75831f17a8, fceffd69017a4353944d63730b8c7cde; no voters:
I20260807 18:21:02.783208 32722 raft_consensus.cc:2781] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:02.783370 32722 raft_consensus.cc:482] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:02.783514 32722 raft_consensus.cc:3037] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.787057 32722 raft_consensus.cc:504] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.787765 32722 leader_election.cc:288] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 election: Requested vote from peers fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809), 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799)
I20260807 18:21:02.788349 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8517b9a6ede483e85e9b5bf3536b400" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceffd69017a4353944d63730b8c7cde"
I20260807 18:21:02.788363 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8517b9a6ede483e85e9b5bf3536b400" candidate_uuid: "d009c5dea31a40649e3aed75831f17a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474091b269de4442b4676075c8228dc2"
I20260807 18:21:02.788589 32226 raft_consensus.cc:3037] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.788645 32615 raft_consensus.cc:3037] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:02.790850 32615 raft_consensus.cc:2454] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 1.
I20260807 18:21:02.791399 32324 leader_election.cc:302] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 474091b269de4442b4676075c8228dc2, d009c5dea31a40649e3aed75831f17a8; no voters:
I20260807 18:21:02.791754 32722 raft_consensus.cc:2781] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:02.792088 32722 raft_consensus.cc:686] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [term 1 LEADER]: Becoming Leader. State: Replica: d009c5dea31a40649e3aed75831f17a8, State: Running, Role: LEADER
I20260807 18:21:02.792145 32226 raft_consensus.cc:2454] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d009c5dea31a40649e3aed75831f17a8 in term 1.
I20260807 18:21:02.792438 32722 consensus_queue.cc:237] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } }
I20260807 18:21:02.794788 32733 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: TxnStatusTablet state changed. Reason: New leader d009c5dea31a40649e3aed75831f17a8. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.796000 32733 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:02.797425 32765 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:21:02.797781 32765 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:21:02.797590 31998 catalog_manager.cc:5495] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d009c5dea31a40649e3aed75831f17a8 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:02.831243 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.824930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.29.156.194:42635)
0807 18:21:02.825204 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.825210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.825237 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:21:02.825329 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.825336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.825434 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.825567 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:02.825987 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.827374 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.828997 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.829708 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.829890 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.830455 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.830570 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.830673 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.830844 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.830930 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:21:02.831337 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.824722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42635 (local address 127.0.0.1:44078)
0807 18:21:02.825008 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.825031 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.825175 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.825615 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.825618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.825627 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:02.825839 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.825847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.827604 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.827611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.828827 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.828839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.830007 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.830012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.830160 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.830239 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.830676 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.830717 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.830812 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.831026 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:21:02.835767 32226 raft_consensus.cc:1261] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Refusing update from remote peer d009c5dea31a40649e3aed75831f17a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.835892 32615 raft_consensus.cc:1261] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Refusing update from remote peer d009c5dea31a40649e3aed75831f17a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.836633 32733 consensus_queue.cc:1035] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.837111 32722 consensus_queue.cc:1035] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.841032 32715 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: TxnStatusTablet state changed. Reason: New leader d009c5dea31a40649e3aed75831f17a8. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.841424 32715 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.841504 32716 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: TxnStatusTablet state changed. Reason: New leader d009c5dea31a40649e3aed75831f17a8. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.841833 32716 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.844851 32722 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.845348 32722 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:02.845551 32727 mvcc.cc:205] Tried to move back new op lower bound from 7315975630168018944 to 7315975630000701440. Current Snapshot: MvccSnapshot[applied={T|T < 7315975630168018944 or (T in {7315975630168018944})}]
I20260807 18:21:02.847563 32722 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.848050 32722 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P d009c5dea31a40649e3aed75831f17a8: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:02.850347 32715 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.850728 32715 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P fceffd69017a4353944d63730b8c7cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.853494 32716 tablet_replica.cc:439] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d009c5dea31a40649e3aed75831f17a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceffd69017a4353944d63730b8c7cde" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42809 } } peers { permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 } } peers { permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 } } }
I20260807 18:21:02.853783 32716 tablet_replica.cc:442] T f8517b9a6ede483e85e9b5bf3536b400 P 474091b269de4442b4676075c8228dc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:02.863325 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.855862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.29.156.193:42809)
0807 18:21:02.856059 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.856065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.856084 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:21:02.856202 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.856207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.856315 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.856551 (+ 236us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:02.856897 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.858510 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.860201 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.860961 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.861120 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.861659 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.862488 (+ 829us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.862662 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.862771 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.862901 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:21:02.882833 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.876061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42635 (local address 127.29.156.193:35271)
0807 18:21:02.876853 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.876869 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.877069 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.877445 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.877450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.877462 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:02.877801 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.877810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.878970 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.878976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.880287 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.880328 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.881520 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.881525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.881655 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.881715 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.882288 (+ 573us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.882300 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.882382 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.882494 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":504,"threads_started":1}
I20260807 18:21:02.882947 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.876007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:35271 (local address 127.29.156.194:42635)
0807 18:21:02.876175 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.876179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.876972 (+ 793us) server_negotiation.cc:407] Connection header received
0807 18:21:02.877236 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.877242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.877326 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.877427 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:02.878015 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.878831 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.880534 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.881301 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.881476 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.881927 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.882145 (+ 218us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.882254 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.882471 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.882577 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:21:02.894164 32615 raft_consensus.cc:1261] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Refusing update from remote peer fceffd69017a4353944d63730b8c7cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.894377 32419 raft_consensus.cc:1261] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Refusing update from remote peer fceffd69017a4353944d63730b8c7cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:02.895079 32750 consensus_queue.cc:1035] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "474091b269de4442b4676075c8228dc2" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.895722 32715 consensus_queue.cc:1035] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d009c5dea31a40649e3aed75831f17a8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:02.902258 32729 mvcc.cc:205] Tried to move back new op lower bound from 7315975630406823936 to 7315975629134839808. Current Snapshot: MvccSnapshot[applied={T|T < 7315975630406823936}]
I20260807 18:21:02.943972 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.936298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:53191 (local address 127.29.156.193:42809)
0807 18:21:02.936557 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:02.936567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:02.937369 (+ 802us) server_negotiation.cc:407] Connection header received
0807 18:21:02.937693 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:02.937699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:02.937785 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:02.937851 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:02.938645 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.939389 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.941931 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.942393 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.942492 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.943144 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.943254 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.943325 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:02.943512 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 18:21:02.943668 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:21:02.944190 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:02.935662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:42809 (local address 127.29.156.194:53191)
0807 18:21:02.937264 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:02.937285 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:02.937509 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:02.938069 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:02.938074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:02.938090 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:02.938456 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.938467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.939558 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.939564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.941785 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:02.941802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:02.942634 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:02.942642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:02.942872 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:02.942950 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:02.943366 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:02.943379 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:21:02.943489 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:21:02.943696 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"thread_start_us":1231,"threads_started":1}
I20260807 18:21:02.995674 31999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:02.995990 31999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:03.000115 31999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0a687cdd5b8445f8df670837befc170: Sending DeleteTablet for 3 replicas of tablet 07420d6aa87a4c9a9b13d71b60191801
I20260807 18:21:03.001549 32193 tablet_service.cc:1514] Processing DeleteTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:55468
I20260807 18:21:03.001965 32740 tablet_replica.cc:330] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: stopping tablet replica
I20260807 18:21:03.002048 32579 tablet_service.cc:1514] Processing DeleteTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39712
I20260807 18:21:03.002226 32740 raft_consensus.cc:2229] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:03.002671 32391 tablet_service.cc:1514] Processing DeleteTablet for tablet 07420d6aa87a4c9a9b13d71b60191801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:02 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:44054
I20260807 18:21:03.003144 32740 raft_consensus.cc:2258] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:03.003602 32741 tablet_replica.cc:330] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: stopping tablet replica
I20260807 18:21:03.003860 32741 raft_consensus.cc:2229] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:03.004108 32741 raft_consensus.cc:2258] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:03.004571 32742 tablet_replica.cc:330] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: stopping tablet replica
I20260807 18:21:03.004832 32742 raft_consensus.cc:2229] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:03.005079 32742 raft_consensus.cc:2258] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:03.008489 32741 ts_tablet_manager.cc:1905] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:03.012619 32740 ts_tablet_manager.cc:1905] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:03.014657 32742 ts_tablet_manager.cc:1905] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:03.020079 32741 ts_tablet_manager.cc:1918] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:03.020321 32741 log.cc:1196] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/07420d6aa87a4c9a9b13d71b60191801
I20260807 18:21:03.020934 32741 ts_tablet_manager.cc:1939] T 07420d6aa87a4c9a9b13d71b60191801 P 474091b269de4442b4676075c8228dc2: Deleting consensus metadata
I20260807 18:21:03.022231 32740 ts_tablet_manager.cc:1918] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:03.022524 32742 ts_tablet_manager.cc:1918] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:03.022570 31970 catalog_manager.cc:4841] TS 474091b269de4442b4676075c8228dc2 (127.29.156.195:35799): tablet 07420d6aa87a4c9a9b13d71b60191801 (table test-table [id=4b830c0c59894238b115ca0e5f386381]) successfully deleted
I20260807 18:21:03.022708 32742 log.cc:1196] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/07420d6aa87a4c9a9b13d71b60191801
I20260807 18:21:03.023187 32742 ts_tablet_manager.cc:1939] T 07420d6aa87a4c9a9b13d71b60191801 P d009c5dea31a40649e3aed75831f17a8: Deleting consensus metadata
I20260807 18:21:03.027792 32740 log.cc:1196] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/07420d6aa87a4c9a9b13d71b60191801
I20260807 18:21:03.028666 32740 ts_tablet_manager.cc:1939] T 07420d6aa87a4c9a9b13d71b60191801 P fceffd69017a4353944d63730b8c7cde: Deleting consensus metadata
I20260807 18:21:03.030704 31972 catalog_manager.cc:4841] TS fceffd69017a4353944d63730b8c7cde (127.29.156.193:42809): tablet 07420d6aa87a4c9a9b13d71b60191801 (table test-table [id=4b830c0c59894238b115ca0e5f386381]) successfully deleted
I20260807 18:21:03.031507 31971 catalog_manager.cc:4841] TS d009c5dea31a40649e3aed75831f17a8 (127.29.156.194:42635): tablet 07420d6aa87a4c9a9b13d71b60191801 (table test-table [id=4b830c0c59894238b115ca0e5f386381]) successfully deleted
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:03.051240 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:03.014376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.29.156.193:42809)
0807 18:21:03.014694 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:03.014701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:03.032255 (+ 17554us) server_negotiation.cc:407] Connection header received
0807 18:21:03.032490 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:03.032496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:03.032587 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:03.032704 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:03.034592 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.035463 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.036887 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.037531 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.037683 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:03.042037 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:03.042053 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:03.042071 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:03.042110 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:03.046704 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.047745 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.047754 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.047757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.047939 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.048368 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.048373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.048376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.048949 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:03.049304 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:03.050570 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0807 18:21:03.050827 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:21:03.060694 32288 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55478
I20260807 18:21:03.101203 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:03.070651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.29.156.254:36521)
0807 18:21:03.070840 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:03.070846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:03.084212 (+ 13366us) server_negotiation.cc:407] Connection header received
0807 18:21:03.084505 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:03.084511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:03.084609 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:03.084690 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:03.087116 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.088534 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.090396 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.091056 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.091211 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:03.094397 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:03.094402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:03.094405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:03.094443 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:03.097772 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.098675 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.098682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.098685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.098829 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.099384 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.099390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.099393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.099823 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:03.100189 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:03.100508 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:21:03.100802 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 18:21:03.101888 31999 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60744
I20260807 18:21:03.104641 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32098
I20260807 18:21:03.124792 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:03.106631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.29.156.193:42809)
0807 18:21:03.107015 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:03.107024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:03.107840 (+ 816us) server_negotiation.cc:407] Connection header received
0807 18:21:03.108041 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:03.108047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:03.108129 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:03.108228 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:03.109556 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.110437 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.111997 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.112740 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.112861 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:03.116554 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:03.116562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:03.116567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:03.116613 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:03.121248 (+ 4635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.122232 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.122238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.122242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.122405 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.122875 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.122880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.122883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.123354 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:03.123601 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:03.124125 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 18:21:03.124423 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
W20260807 18:21:03.126543 32288 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494
W20260807 18:21:03.127242 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32098: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:03.127434 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 32098
I20260807 18:21:03.166095 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32298
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:03.184971 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:03.166961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.29.156.194:42635)
0807 18:21:03.167188 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:03.167211 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:03.167286 (+ 75us) server_negotiation.cc:407] Connection header received
0807 18:21:03.167698 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:03.167703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:03.167772 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:03.167916 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:03.169636 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.170554 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.172465 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.172929 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.173035 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:03.178352 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:03.178367 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:03.178384 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:03.178422 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:03.181913 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.182648 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.182656 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.182658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.182792 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.183187 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.183192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.183200 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.183695 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:03.184012 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:03.184343 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:21:03.184599 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 18:21:03.185642 32473 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094
W20260807 18:21:03.186331 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32298: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:03.186527 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 32298
I20260807 18:21:03.219265 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32484
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[31921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:03.234889 321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:03.220145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39722 (local address 127.29.156.195:35799)
0807 18:21:03.220691 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:03.220696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:03.220711 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:21:03.220766 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:03.220770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:03.220834 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:03.220978 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:03.222229 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.222927 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.224260 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:03.224673 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:03.224817 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:03.229036 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:03.229051 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:03.229068 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:03.229129 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:03.232286 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.232933 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.232941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.232943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.233095 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:03.233461 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:03.233466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:03.233468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:03.233875 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:03.234172 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:03.234407 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 18:21:03.234588 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":283,"threads_started":1}
W20260807 18:21:03.235751 32669 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39722
W20260807 18:21:03.236397 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 32484: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:03.236577 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 32484
I20260807 18:21:03.264899 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 31944
I20260807 18:21:03.266233 32053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:03.395320 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 31944
2026-08-07T18:21:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5234 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[325](info): set up 2 sockets
Aug 07 18:21:03 dist-test-slave-sb2z krb5kdc[325](info): commencing operation
krb5kdc: starting...
W20260807 18:21:04.766776 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:04 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:05.008292 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43477
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:05.363845 343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:05.364207 343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:05.364375 343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:05.371919 343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:05.372076 343 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:21:05.372169 343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:05.372287 343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:05.372373 343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:05.372467 343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:05.382858 343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43477
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:05.384366 343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:05.386142 343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:05.400663 352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:05.400763 355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:05.400789 350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:05.401821 343 server_base.cc:1034] running on GCE node
I20260807 18:21:05.402640 343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:05.404184 343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:05.405481 343 hybrid_clock.cc:648] HybridClock initialized: now 1786126865405407 us; error 43 us; skew 500 ppm
I20260807 18:21:05.405858 343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:05 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126865, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:05.409301 343 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:05.410773 343 webserver.cc:460] Webserver started at http://127.29.156.254:44743/ using document root <none> and password file <none>
I20260807 18:21:05.411473 343 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:05.411651 343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:05.412014 343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:05.415172 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "5402b5c42a3344a993ba185350f85b87"
format_stamp: "Formatted at 2026-08-07 18:21:05 on dist-test-slave-sb2z"
server_key: "95183a8216b3e5bd83b32200a14f20dd"
server_key_iv: "3b81c30fd76a39d5f1d3eb3fabb1b607"
server_key_version: "encryptionkey@0"
I20260807 18:21:05.416061 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "5402b5c42a3344a993ba185350f85b87"
format_stamp: "Formatted at 2026-08-07 18:21:05 on dist-test-slave-sb2z"
server_key: "95183a8216b3e5bd83b32200a14f20dd"
server_key_iv: "3b81c30fd76a39d5f1d3eb3fabb1b607"
server_key_version: "encryptionkey@0"
I20260807 18:21:05.421669 343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:21:05.425637 366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:05.427098 343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:05.427314 343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "5402b5c42a3344a993ba185350f85b87"
format_stamp: "Formatted at 2026-08-07 18:21:05 on dist-test-slave-sb2z"
server_key: "95183a8216b3e5bd83b32200a14f20dd"
server_key_iv: "3b81c30fd76a39d5f1d3eb3fabb1b607"
server_key_version: "encryptionkey@0"
I20260807 18:21:05.427588 343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:05.450071 343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:05.454859 343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:05.455120 343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:05.478879 343 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43477
I20260807 18:21:05.478901 450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43477 every 8 connection(s)
I20260807 18:21:05.480481 343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:05.483374 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 343
I20260807 18:21:05.483664 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:05.484433 30323 external_mini_cluster.cc:1428] Setting key bf3210a83c99cf97a999082a8b650af7
I20260807 18:21:05.484709 451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:05.499331 451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Bootstrap starting.
Aug 07 18:21:05 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:05.503334 451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:05.504928 451 log.cc:824] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:05.508669 451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: No bootstrap required, opened a new log
I20260807 18:21:05.511466 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:05.486670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.29.156.254:43477)
0807 18:21:05.487596 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:05.487617 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:05.487667 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:21:05.489803 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:05.489834 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:05.490379 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:05.490841 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:05.492426 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:05.494410 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:05.495936 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:05.497093 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:05.497291 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:05.502161 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:05.502177 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:05.502197 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:05.502247 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:05.506343 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:05.507218 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:05.507227 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:05.507236 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:05.507476 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:05.507960 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:05.507965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:05.507968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:05.508681 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:05.508985 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:05.509486 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 18:21:05.509949 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":396,"threads_started":1}
I20260807 18:21:05.517886 451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } }
I20260807 18:21:05.518237 451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:05.518386 451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5402b5c42a3344a993ba185350f85b87, State: Initialized, Role: FOLLOWER
I20260807 18:21:05.519065 451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } }
I20260807 18:21:05.519388 451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:05.519579 451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:05.519774 451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:05.522156 451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } }
I20260807 18:21:05.522768 451 leader_election.cc:302] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5402b5c42a3344a993ba185350f85b87; no voters:
I20260807 18:21:05.523411 451 leader_election.cc:288] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:05.523690 459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:05.524623 459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [term 1 LEADER]: Becoming Leader. State: Replica: 5402b5c42a3344a993ba185350f85b87, State: Running, Role: LEADER
I20260807 18:21:05.525211 459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } }
I20260807 18:21:05.525806 451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:05.528591 460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5402b5c42a3344a993ba185350f85b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } } }
I20260807 18:21:05.528538 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5402b5c42a3344a993ba185350f85b87. Latest consensus state: current_term: 1 leader_uuid: "5402b5c42a3344a993ba185350f85b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5402b5c42a3344a993ba185350f85b87" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43477 } } }
I20260807 18:21:05.528987 460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:05.529074 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:05.530481 472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:05.536417 472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:05.547020 472 catalog_manager.cc:1347] Generated new cluster ID: b1947ddd1b214241bee51fa83ad7f565
I20260807 18:21:05.547174 472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:05.559643 472 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:05.561116 472 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:05.569311 472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Generated new TSK 0
I20260807 18:21:05.570237 472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:05.638831 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43477
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:05.988567 495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:05.988862 495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:05.989112 495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:05.996374 495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:05.996522 495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:05.996687 495 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:21:05.996769 495 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:05.996888 495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:06.007121 495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:43477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:06.008567 495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:06.010051 495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:06.024266 501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:06.024385 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:06.024412 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:06.025337 495 server_base.cc:1034] running on GCE node
I20260807 18:21:06.026031 495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:06.027135 495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:06.028369 495 hybrid_clock.cc:648] HybridClock initialized: now 1786126866028285 us; error 80 us; skew 500 ppm
I20260807 18:21:06.028725 495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:06 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:06.037942 495 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:06.039798 495 webserver.cc:460] Webserver started at http://127.29.156.193:44405/ using document root <none> and password file <none>
I20260807 18:21:06.040606 495 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:06.040729 495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:06.041079 495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:06.044512 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "001481e32beb42e1be1020b7991d0a63"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "082611e0a4bd2bbb9c0520311382dd0d"
server_key_iv: "6b8e29d35464437d6c3be7dae2017029"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.045361 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "001481e32beb42e1be1020b7991d0a63"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "082611e0a4bd2bbb9c0520311382dd0d"
server_key_iv: "6b8e29d35464437d6c3be7dae2017029"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.051285 495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 18:21:06.055122 514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:06.056780 495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:06.057086 495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "001481e32beb42e1be1020b7991d0a63"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "082611e0a4bd2bbb9c0520311382dd0d"
server_key_iv: "6b8e29d35464437d6c3be7dae2017029"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.057320 495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:06.076282 495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:06.080983 495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:06.081311 495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:06.084220 495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:06.084372 495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:06.084510 495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:06.084610 495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:06 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:06.111433 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.084880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:39145 (local address 127.29.156.254:43477)
0807 18:21:06.085591 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:06.085599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:06.086743 (+ 1144us) server_negotiation.cc:407] Connection header received
0807 18:21:06.089872 (+ 3129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:06.089881 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:06.090003 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:06.090139 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:06.092692 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.093518 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.096468 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.096880 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.097001 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.102866 (+ 5865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:06.102876 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:06.102884 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:06.102944 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:06.107199 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.108550 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.108558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.108564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.108746 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.109471 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.109478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.109484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.109819 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:06.109999 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:06.110775 (+ 776us) server_negotiation.cc:299] Negotiation successful
0807 18:21:06.111035 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":424,"threads_started":1}
I20260807 18:21:06.113215 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.085614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.193:39145)
0807 18:21:06.086535 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:06.086599 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:06.089455 (+ 2856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:06.090577 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:06.090592 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:06.091643 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:06.092493 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.092513 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.093717 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.093731 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.096281 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.096298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.097189 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.097199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.097588 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.098518 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:06.098543 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:06.102530 (+ 3987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:06.107426 (+ 4896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.107441 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.107467 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.108341 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.108916 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.108922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.108926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.109270 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.109960 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:06.109969 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:06.110374 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 18:21:06.110657 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 18:21:06.111249 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":361,"threads_started":1}
I20260807 18:21:06.135499 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.119790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40359 (local address 127.29.156.254:43477)
0807 18:21:06.120028 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:06.120037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:06.120749 (+ 712us) server_negotiation.cc:407] Connection header received
0807 18:21:06.121178 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:06.121185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:06.121275 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:06.121376 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:06.122948 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.123738 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.125982 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.126518 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.126659 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.129209 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:06.129215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:06.129219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:06.129260 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:06.132088 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.133162 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.133168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.133172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.133304 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.133808 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.133814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.133820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.134185 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:06.134377 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:06.135023 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 18:21:06.135223 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:21:06.136121 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.119735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.193:40359)
0807 18:21:06.120600 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:06.120656 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:06.120984 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:06.121409 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:06.121414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:06.122173 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:06.122753 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.122781 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.123883 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.123889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.125805 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.125830 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.126709 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.126716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.126879 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.127619 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:06.127638 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:06.128928 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:06.132229 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.132235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.132239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.133002 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.133443 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.133447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.133452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.133648 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.134335 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:06.134343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:06.134640 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 18:21:06.134850 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 18:21:06.135329 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":536,"threads_started":1}
I20260807 18:21:06.142349 495 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:43641
I20260807 18:21:06.142387 683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:43641 every 8 connection(s)
I20260807 18:21:06.143625 495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:06.145968 684 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43477
I20260807 18:21:06.146221 684 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:06.146808 684 heartbeater.cc:511] Master 127.29.156.254:43477 requested a full tablet report, sending...
I20260807 18:21:06.148859 388 ts_manager.cc:194] Registered new tserver with Master: 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641)
I20260807 18:21:06.150403 388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:39145
I20260807 18:21:06.153617 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 495
I20260807 18:21:06.153780 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:06.154449 30323 external_mini_cluster.cc:1428] Setting key 220c3bca8e970191b62f0a1b39a8f727
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:06.207623 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43477
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:06.551971 688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:06.552337 688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:06.552525 688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:06.560103 688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:06.560253 688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:06.560396 688 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:21:06.560487 688 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:06.560623 688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:06.570864 688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:43477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:06.572292 688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:06.573618 688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:06.587555 700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:06.587492 701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:06.587452 703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:06.587918 688 server_base.cc:1034] running on GCE node
I20260807 18:21:06.588835 688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:06.589763 688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:06.590997 688 hybrid_clock.cc:648] HybridClock initialized: now 1786126866590892 us; error 98 us; skew 500 ppm
I20260807 18:21:06.591280 688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:06 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:06.594504 688 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:06.596369 688 webserver.cc:460] Webserver started at http://127.29.156.194:41589/ using document root <none> and password file <none>
I20260807 18:21:06.597102 688 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:06.597237 688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:06.597600 688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:06.600739 688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "0039e197a64d402c81557d8bb736d61a"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "3ba8ceedd3f2693e7d507a56deff113f"
server_key_iv: "f203b7a4c5af50edb2d0409956a6ce02"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.601542 688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "0039e197a64d402c81557d8bb736d61a"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "3ba8ceedd3f2693e7d507a56deff113f"
server_key_iv: "f203b7a4c5af50edb2d0409956a6ce02"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.607107 688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:21:06.611197 717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:06.612990 688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:06.613232 688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "0039e197a64d402c81557d8bb736d61a"
format_stamp: "Formatted at 2026-08-07 18:21:06 on dist-test-slave-sb2z"
server_key: "3ba8ceedd3f2693e7d507a56deff113f"
server_key_iv: "f203b7a4c5af50edb2d0409956a6ce02"
server_key_version: "encryptionkey@0"
I20260807 18:21:06.613482 688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:06.640300 688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:06.644937 688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:06.645290 688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:06.648418 688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:06.648554 688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:06.648691 688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:06.648798 688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:06 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:06.676648 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.648938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35535 (local address 127.29.156.254:43477)
0807 18:21:06.649745 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:06.649750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:06.650694 (+ 944us) server_negotiation.cc:407] Connection header received
0807 18:21:06.652936 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:06.652942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:06.653069 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:06.653219 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:06.655997 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.657159 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.660048 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.660580 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.660728 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.666644 (+ 5916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:06.666650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:06.666653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:06.666693 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:06.670528 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.671795 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.671804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.671809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.671989 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.672704 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.672710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.672714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.673162 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:06.673361 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:06.675786 (+ 2425us) server_negotiation.cc:299] Negotiation successful
0807 18:21:06.676117 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":575,"threads_started":1}
I20260807 18:21:06.677409 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.649675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.194:35535)
0807 18:21:06.650521 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:06.650563 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:06.652599 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:06.653556 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:06.653574 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:06.654573 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:06.655742 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.655777 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.657363 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.657367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.659866 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.659887 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.661098 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.661113 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.661382 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.662311 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:06.662346 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:06.666351 (+ 4005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:06.670694 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.670709 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.670736 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.671596 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.672140 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.672151 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.672155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.672536 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.673343 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:06.673356 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:06.674039 (+ 683us) client_negotiation.cc:769] Sending connection context
0807 18:21:06.674465 (+ 426us) client_negotiation.cc:240] Negotiation successful
0807 18:21:06.675246 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":370,"threads_started":1}
I20260807 18:21:06.698985 688 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43463
I20260807 18:21:06.699019 865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43463 every 8 connection(s)
I20260807 18:21:06.700855 688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:06.702206 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 688
I20260807 18:21:06.702433 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:06.703161 30323 external_mini_cluster.cc:1428] Setting key 1182e4c7f9d84314577a507cf4d53b15
I20260807 18:21:06.703663 866 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43477
I20260807 18:21:06.704020 866 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:06.705082 866 heartbeater.cc:511] Master 127.29.156.254:43477 requested a full tablet report, sending...
I20260807 18:21:06.706961 388 ts_manager.cc:194] Registered new tserver with Master: 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463)
I20260807 18:21:06.708333 388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:35535
I20260807 18:21:06.709589 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.684294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:33951 (local address 127.29.156.254:43477)
0807 18:21:06.684515 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:06.684525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:06.691741 (+ 7216us) server_negotiation.cc:407] Connection header received
0807 18:21:06.692124 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:06.692129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:06.692205 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:06.692301 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:06.694344 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.695690 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.698146 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.698914 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.699032 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.701655 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:06.701664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:06.701671 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:06.701726 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:06.705934 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.706956 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.706962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.706965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.707132 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:06.707671 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:06.707676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:06.707680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:06.708150 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:06.708286 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:06.708973 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 18:21:06.709238 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:21:06.710741 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:06.684436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.194:33951)
0807 18:21:06.691444 (+ 7008us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:06.691476 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:06.691906 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:06.692385 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:06.692394 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:06.693272 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:06.694100 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.694125 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.695928 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.695936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.697925 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:06.697941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:06.699349 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:06.699356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:06.699506 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:06.700213 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:06.700236 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:06.701375 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:06.706125 (+ 4750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.706132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.706136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.706791 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.707307 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:06.707312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:06.707316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:06.707496 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:06.708411 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:06.708417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:06.708653 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 18:21:06.709794 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0807 18:21:06.710208 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6768,"thread_start_us":6555,"threads_started":1}
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:06.761847 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43477
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:07.129175 871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:07.129479 871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:07.129608 871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:07.138226 871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:07.138355 871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:07.138453 871 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:21:07.138523 871 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:21:07.138602 871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:07.152868 871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:43477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:07.153286 684 heartbeater.cc:503] Master 127.29.156.254:43477 was elected leader, sending a full tablet report...
I20260807 18:21:07.154529 871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:07.156364 871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:07.171633 879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:07.171773 882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:07.171974 886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:07.172868 871 server_base.cc:1034] running on GCE node
I20260807 18:21:07.173596 871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:07.174639 871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:07.175895 871 hybrid_clock.cc:648] HybridClock initialized: now 1786126867175816 us; error 63 us; skew 500 ppm
I20260807 18:21:07.176278 871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:07 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126867, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:07.179804 871 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:07.181540 871 webserver.cc:460] Webserver started at http://127.29.156.195:45205/ using document root <none> and password file <none>
I20260807 18:21:07.182327 871 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:07.182492 871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:07.182845 871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:07.186146 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "00d647fb147141ad9a6f448f6c88aba7"
format_stamp: "Formatted at 2026-08-07 18:21:07 on dist-test-slave-sb2z"
server_key: "fe1acc29cce4b326a5c577ea742e4460"
server_key_iv: "48b109bfba6635a1066375107139a32b"
server_key_version: "encryptionkey@0"
I20260807 18:21:07.187021 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "00d647fb147141ad9a6f448f6c88aba7"
format_stamp: "Formatted at 2026-08-07 18:21:07 on dist-test-slave-sb2z"
server_key: "fe1acc29cce4b326a5c577ea742e4460"
server_key_iv: "48b109bfba6635a1066375107139a32b"
server_key_version: "encryptionkey@0"
I20260807 18:21:07.193078 871 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:21:07.196772 896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:07.198274 871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:07.198412 871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "00d647fb147141ad9a6f448f6c88aba7"
format_stamp: "Formatted at 2026-08-07 18:21:07 on dist-test-slave-sb2z"
server_key: "fe1acc29cce4b326a5c577ea742e4460"
server_key_iv: "48b109bfba6635a1066375107139a32b"
server_key_version: "encryptionkey@0"
I20260807 18:21:07.198594 871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:07.221252 871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:07.225812 871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:07.226145 871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:07.229296 871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:07.229451 871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:07.229590 871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:07.229709 871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:21:07 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126867, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:07.262460 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.231237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.195:57721)
0807 18:21:07.232320 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.232468 (+ 148us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.235415 (+ 2947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.236829 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.236846 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.237918 (+ 1072us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:07.239001 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.239030 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.240697 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.240701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.243231 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.243252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.244855 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.244863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.245289 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.246450 (+ 1161us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:07.246505 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:07.250463 (+ 3958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:07.255495 (+ 5032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.255503 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.255518 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.256493 (+ 975us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.257093 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.257098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.257101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.257343 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.258308 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:07.258319 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:07.259019 (+ 700us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.259465 (+ 446us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.260259 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":466,"threads_started":1}
I20260807 18:21:07.263713 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.230519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:57721 (local address 127.29.156.254:43477)
0807 18:21:07.231058 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.231065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.232595 (+ 1530us) server_negotiation.cc:407] Connection header received
0807 18:21:07.235903 (+ 3308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.235914 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.236030 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.236130 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:07.239231 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.240443 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.243462 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.244271 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.244461 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.250876 (+ 6415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:07.250888 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:07.250896 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:07.250958 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:07.255313 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.256673 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.256681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.256687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.256905 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.257577 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.257585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.257592 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.257960 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:07.258179 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.263117 (+ 4938us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.263321 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":264,"threads_started":1}
I20260807 18:21:07.280427 871 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:41535
I20260807 18:21:07.280472 1048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:41535 every 8 connection(s)
I20260807 18:21:07.281821 871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:07.284276 1049 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43477
I20260807 18:21:07.284538 1049 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:07.285355 1049 heartbeater.cc:511] Master 127.29.156.254:43477 requested a full tablet report, sending...
I20260807 18:21:07.287014 388 ts_manager.cc:194] Registered new tserver with Master: 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:07.287093 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 871
I20260807 18:21:07.287269 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:07.287853 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.270936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:53513 (local address 127.29.156.254:43477)
0807 18:21:07.271172 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.271181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.272201 (+ 1020us) server_negotiation.cc:407] Connection header received
0807 18:21:07.272449 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.272454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.272527 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.272617 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:07.274226 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.275030 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.278017 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.278539 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.278648 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.281382 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:07.281388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:07.281391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:07.281426 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:07.284843 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.285558 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.285564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.285568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.285725 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.286119 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.286124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.286127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.286501 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:07.286703 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.287202 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.287460 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:21:07.287880 30323 external_mini_cluster.cc:1428] Setting key d430e603e6ce990c8fef5dc05e046e4a
I20260807 18:21:07.287879 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.271034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.29.156.195:53513)
0807 18:21:07.272050 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.272073 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.272280 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.272694 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.272701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.273489 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:07.273998 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.274017 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.275201 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.275209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.277742 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.277764 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.278760 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.278770 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.279025 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.279936 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:07.279971 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:07.281152 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:07.284973 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.284977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.284980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.285438 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.285846 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.285852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.285856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.285994 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.286644 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:07.286649 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:07.286885 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.287043 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.287370 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":582,"threads_started":1}
I20260807 18:21:07.288214 388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:57721
I20260807 18:21:07.291838 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:21:07 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:21:07 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:07.354856 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.339094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.29.156.254:43477)
0807 18:21:07.339301 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.339306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.339779 (+ 473us) server_negotiation.cc:407] Connection header received
0807 18:21:07.340011 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.340015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.340091 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.340190 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:07.341787 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.342597 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.344149 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.344592 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.344699 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.349200 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:07.349206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:07.349209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:07.349246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:07.352222 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.352893 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.352897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.352900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.353050 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.353516 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.353520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.353523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.353843 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:07.354129 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.354367 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.354563 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:21:07.359867 388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:07.368012 388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:07.394344 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.382833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.29.156.193:43641)
0807 18:21:07.384622 (+ 1789us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.384636 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.384677 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:21:07.384817 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.384831 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.385185 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.385448 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.386354 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.387822 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.391115 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.392879 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.393135 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.393341 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.393822 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.394019 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468,"thread_start_us":1286,"threads_started":1}
I20260807 18:21:07.394464 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.382577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.0.0.1:39572)
0807 18:21:07.383412 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.383467 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.383713 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.385476 (+ 1763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.385491 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.385528 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.386079 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.386094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.388009 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.388017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.390909 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.390927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.393168 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.393177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.393506 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.393586 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.393738 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.393964 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":30,"thread_start_us":402,"threads_started":1}
I20260807 18:21:07.396719 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.384471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.29.156.194:43463)
0807 18:21:07.387099 (+ 2628us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.387112 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.387152 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:21:07.387282 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.387295 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.387665 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.387923 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.388835 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.390472 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.393015 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.394610 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.394827 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.395042 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.396096 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.396281 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2410,"thread_start_us":2235,"threads_started":1}
I20260807 18:21:07.397591 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.383401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43463 (local address 127.0.0.1:43420)
0807 18:21:07.384826 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.384854 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.384992 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.387943 (+ 2951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.387951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.387988 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.388611 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.388625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.390678 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.390684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.392823 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.392841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.395460 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.395472 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.395709 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.395751 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.395860 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.397181 (+ 1321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1268,"thread_start_us":1902,"threads_started":1}
I20260807 18:21:07.398500 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.383874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:41535 (local address 127.0.0.1:58280)
0807 18:21:07.388758 (+ 4884us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.388785 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.389003 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.389975 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.389981 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.390002 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.390378 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.390385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.392628 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.392634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.395090 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.395107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.397589 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.397597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.397826 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.397869 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.397979 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.398118 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4699,"mutex_wait_us":37,"thread_start_us":4498,"threads_started":1}
I20260807 18:21:07.398793 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.384952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.29.156.195:41535)
0807 18:21:07.386552 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.386566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.388888 (+ 2322us) server_negotiation.cc:407] Connection header received
0807 18:21:07.389219 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.389229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.389487 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.389679 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.390595 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.392427 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.395294 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.396976 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.397200 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.397394 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.398058 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.398235 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1354,"thread_start_us":1107,"threads_started":1}
I20260807 18:21:07.399312 589 tablet_service.cc:1467] Processing CreateTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 (DEFAULT_TABLE table=test-table [id=33e1edb1ebee45a69a6d37719da46751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.401026 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7853d3cbe144eb9a2cbae52b9b2163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.402661 962 tablet_service.cc:1467] Processing CreateTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 (DEFAULT_TABLE table=test-table [id=33e1edb1ebee45a69a6d37719da46751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.402657 779 tablet_service.cc:1467] Processing CreateTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 (DEFAULT_TABLE table=test-table [id=33e1edb1ebee45a69a6d37719da46751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.404302 779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7853d3cbe144eb9a2cbae52b9b2163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.404389 962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7853d3cbe144eb9a2cbae52b9b2163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.417971 1077 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Bootstrap starting.
I20260807 18:21:07.420284 1078 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Bootstrap starting.
I20260807 18:21:07.422459 1079 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Bootstrap starting.
I20260807 18:21:07.423956 1077 tablet_bootstrap.cc:654] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.424528 1078 tablet_bootstrap.cc:654] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.425503 1077 log.cc:824] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:07.425800 1079 tablet_bootstrap.cc:654] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.425921 1078 log.cc:824] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:07.427263 1079 log.cc:824] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:07.429414 1078 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: No bootstrap required, opened a new log
I20260807 18:21:07.429411 1077 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: No bootstrap required, opened a new log
I20260807 18:21:07.429735 1078 ts_tablet_manager.cc:1397] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:21:07.429740 1077 ts_tablet_manager.cc:1397] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:21:07.430601 1079 tablet_bootstrap.cc:492] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: No bootstrap required, opened a new log
I20260807 18:21:07.430931 1079 ts_tablet_manager.cc:1397] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:21:07.439451 1079 raft_consensus.cc:348] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.440016 1079 raft_consensus.cc:374] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.440129 1079 raft_consensus.cc:729] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.440971 1079 consensus_queue.cc:260] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.442153 1077 raft_consensus.cc:348] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.442590 866 heartbeater.cc:503] Master 127.29.156.254:43477 was elected leader, sending a full tablet report...
I20260807 18:21:07.442648 1077 raft_consensus.cc:374] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.442756 1077 raft_consensus.cc:729] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.442150 1078 raft_consensus.cc:348] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.443321 1078 raft_consensus.cc:374] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.443409 1079 ts_tablet_manager.cc:1428] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 18:21:07.443595 1078 raft_consensus.cc:729] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.443509 1077 consensus_queue.cc:260] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.444437 1078 consensus_queue.cc:260] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.444890 1049 heartbeater.cc:503] Master 127.29.156.254:43477 was elected leader, sending a full tablet report...
I20260807 18:21:07.445528 1077 ts_tablet_manager.cc:1428] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260807 18:21:07.447266 1078 ts_tablet_manager.cc:1428] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
W20260807 18:21:07.464000 867 tablet.cc:2369] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:07.529614 1084 raft_consensus.cc:482] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:07.530040 1084 raft_consensus.cc:504] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.532169 1084 leader_election.cc:288] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
W20260807 18:21:07.533777 1050 tablet.cc:2369] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:07.540530 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.532549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:49645 (local address 127.29.156.193:43641)
0807 18:21:07.532870 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.532875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.533355 (+ 480us) server_negotiation.cc:407] Connection header received
0807 18:21:07.533656 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.533666 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.533791 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.533878 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.534892 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.536026 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.538180 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.539562 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.539740 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.539798 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.540200 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.540303 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:21:07.540975 1087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.532460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.29.156.194:49645)
0807 18:21:07.533231 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.533261 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.533411 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.534236 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.534245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.534274 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.534734 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.534749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.536177 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.536184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.537996 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.538012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.539827 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.539833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.540022 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.540061 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.540166 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.540305 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":30,"thread_start_us":386,"threads_started":1}
I20260807 18:21:07.541633 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.533794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:43577 (local address 127.29.156.195:41535)
0807 18:21:07.534419 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.534427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.534550 (+ 123us) server_negotiation.cc:407] Connection header received
0807 18:21:07.534808 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.534816 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.534960 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.535042 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.535794 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.536865 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.539331 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.540526 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.540705 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.540769 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.541251 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.541381 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454}
I20260807 18:21:07.541714 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.533403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:41535 (local address 127.29.156.194:43577)
0807 18:21:07.534427 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.534451 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.534608 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.535175 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.535180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.535212 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.535620 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.535634 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.537073 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.537078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.539157 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.539175 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.540784 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.540792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.540973 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.541105 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.541218 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.541337 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":1,"thread_start_us":605,"threads_started":1}
I20260807 18:21:07.542224 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7853d3cbe144eb9a2cbae52b9b2163" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63" is_pre_election: true
I20260807 18:21:07.542901 619 raft_consensus.cc:2454] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 0.
I20260807 18:21:07.542860 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7853d3cbe144eb9a2cbae52b9b2163" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7" is_pre_election: true
I20260807 18:21:07.543354 989 raft_consensus.cc:2454] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 0.
I20260807 18:21:07.543726 720 leader_election.cc:302] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 0039e197a64d402c81557d8bb736d61a; no voters:
I20260807 18:21:07.544247 1084 raft_consensus.cc:2781] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:07.544446 1084 raft_consensus.cc:482] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:07.544587 1084 raft_consensus.cc:3037] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.547165 1084 raft_consensus.cc:504] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.547968 1084 leader_election.cc:288] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 election: Requested vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:07.548511 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7853d3cbe144eb9a2cbae52b9b2163" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63"
I20260807 18:21:07.548707 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7853d3cbe144eb9a2cbae52b9b2163" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7"
I20260807 18:21:07.548779 619 raft_consensus.cc:3037] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.548988 989 raft_consensus.cc:3037] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.551294 619 raft_consensus.cc:2454] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 1.
I20260807 18:21:07.551311 989 raft_consensus.cc:2454] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 1.
I20260807 18:21:07.551826 720 leader_election.cc:302] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 0039e197a64d402c81557d8bb736d61a; no voters:
I20260807 18:21:07.552217 1084 raft_consensus.cc:2781] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:07.552762 1084 raft_consensus.cc:686] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 1 LEADER]: Becoming Leader. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Running, Role: LEADER
I20260807 18:21:07.553293 1084 consensus_queue.cc:237] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } }
I20260807 18:21:07.558427 387 catalog_manager.cc:5495] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0039e197a64d402c81557d8bb736d61a (127.29.156.194). New cstate: current_term: 1 leader_uuid: "0039e197a64d402c81557d8bb736d61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } health_report { overall_health: HEALTHY } } }
Aug 07 18:21:07 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126865, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:07.642002 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.626625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43477 (local address 127.0.0.1:60326)
0807 18:21:07.627483 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.627501 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.627707 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.628127 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.628131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.628896 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:07.629178 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.629188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.630373 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.630377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.631633 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.631646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.632532 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.632539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.632703 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.633509 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:07.633527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:07.636228 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:07.639308 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.639320 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.639326 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.639933 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.640351 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:07.640354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:07.640357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:07.640541 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:07.641070 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:07.641076 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:07.641239 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.641381 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.641625 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":532,"threads_started":1}
I20260807 18:21:07.642138 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.626462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.29.156.254:43477)
0807 18:21:07.626682 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.626692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.627663 (+ 971us) server_negotiation.cc:407] Connection header received
0807 18:21:07.627819 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.627824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.627892 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.628053 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:07.629346 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.630228 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.631767 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.632327 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.632437 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.636420 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:07.636426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:07.636429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:07.636463 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:07.639164 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.640084 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.640088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.640091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.640234 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:07.640652 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:07.640656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:07.640658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:07.640944 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:07.641062 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.641484 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.641764 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":69}
W20260807 18:21:07.646368 685 tablet.cc:2369] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:07.648020 387 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:60326:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:21:07.649149 387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:07.656423 589 tablet_service.cc:1467] Processing CreateTablet for tablet 39d6178685e44ddc9e93adc34592078a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c26fc2e9d42483c8621d582d64c0e60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:07.657407 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d6178685e44ddc9e93adc34592078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.656936 779 tablet_service.cc:1467] Processing CreateTablet for tablet 39d6178685e44ddc9e93adc34592078a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c26fc2e9d42483c8621d582d64c0e60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:07.656936 962 tablet_service.cc:1467] Processing CreateTablet for tablet 39d6178685e44ddc9e93adc34592078a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c26fc2e9d42483c8621d582d64c0e60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:21:07.657936 962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d6178685e44ddc9e93adc34592078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.657936 779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d6178685e44ddc9e93adc34592078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.663357 1078 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: Bootstrap starting.
I20260807 18:21:07.664986 1078 tablet_bootstrap.cc:654] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.666458 1079 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: Bootstrap starting.
I20260807 18:21:07.666934 1077 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: Bootstrap starting.
I20260807 18:21:07.668171 1079 tablet_bootstrap.cc:654] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.668438 1078 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: No bootstrap required, opened a new log
I20260807 18:21:07.668601 1078 ts_tablet_manager.cc:1397] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:21:07.668903 1077 tablet_bootstrap.cc:654] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.669538 1078 raft_consensus.cc:348] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.669806 1078 raft_consensus.cc:374] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.669910 1078 raft_consensus.cc:729] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.670239 1078 consensus_queue.cc:260] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.670935 1079 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: No bootstrap required, opened a new log
I20260807 18:21:07.670933 1078 ts_tablet_manager.cc:1428] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:21:07.671092 1079 ts_tablet_manager.cc:1397] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:21:07.670855 1086 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.671388 1086 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.671876 1079 raft_consensus.cc:348] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.672115 1079 raft_consensus.cc:374] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.672228 1079 raft_consensus.cc:729] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.672469 1079 consensus_queue.cc:260] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.673450 1079 ts_tablet_manager.cc:1428] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:21:07.672888 1084 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.673991 1084 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.676091 1077 tablet_bootstrap.cc:492] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: No bootstrap required, opened a new log
I20260807 18:21:07.676272 1077 ts_tablet_manager.cc:1397] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 18:21:07.677038 1077 raft_consensus.cc:348] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.677323 1077 raft_consensus.cc:374] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.677433 1077 raft_consensus.cc:729] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.677610 1086 raft_consensus.cc:482] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:07.677706 1077 consensus_queue.cc:260] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.677798 1086 raft_consensus.cc:504] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.678350 1077 ts_tablet_manager.cc:1428] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:21:07.678265 1085 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.678700 1085 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.679701 1086 leader_election.cc:288] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:07.689415 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.680289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:41535 (local address 127.29.156.193:49903)
0807 18:21:07.681207 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.681227 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.681381 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.681769 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.681777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.681805 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.682341 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.682357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.684146 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.684151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.686702 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.686717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.688164 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.688171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.688774 (+ 603us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.688821 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.688932 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.689065 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":136,"thread_start_us":1875,"threads_started":1}
I20260807 18:21:07.689507 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.681051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:49903 (local address 127.29.156.195:41535)
0807 18:21:07.681218 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.681226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.681322 (+ 96us) server_negotiation.cc:407] Connection header received
0807 18:21:07.681538 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.681545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.681629 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.681739 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.682949 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.683971 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.686870 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.687929 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.688126 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.688212 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.689074 (+ 862us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.689232 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:21:07.690068 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.680130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43463 (local address 127.29.156.193:40141)
0807 18:21:07.681268 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.681286 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.681436 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.681965 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.681973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.682009 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:07.682492 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.682503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.684006 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.684015 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.686702 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.686717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.689188 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.689195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.689370 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.689406 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.689513 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.689666 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":147,"thread_start_us":439,"threads_started":1}
I20260807 18:21:07.690268 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.680590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40141 (local address 127.29.156.194:43463)
0807 18:21:07.680792 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.680800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.681357 (+ 557us) server_negotiation.cc:407] Connection header received
0807 18:21:07.681619 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.681624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.681711 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.681800 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:07.682663 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.683866 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.687664 (+ 3798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.688738 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.688925 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.688998 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.689846 (+ 848us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.689990 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:21:07.690171 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d6178685e44ddc9e93adc34592078a" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7" is_pre_election: true
I20260807 18:21:07.690462 989 raft_consensus.cc:2454] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 0.
I20260807 18:21:07.691102 520 leader_election.cc:302] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:07.691040 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d6178685e44ddc9e93adc34592078a" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a" is_pre_election: true
I20260807 18:21:07.691565 1086 raft_consensus.cc:2781] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:07.691464 803 raft_consensus.cc:2454] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 0.
I20260807 18:21:07.691696 1086 raft_consensus.cc:482] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:07.691810 1086 raft_consensus.cc:3037] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.693954 1086 raft_consensus.cc:504] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.694567 1086 leader_election.cc:288] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 election: Requested vote from peers 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:07.695142 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d6178685e44ddc9e93adc34592078a" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7"
I20260807 18:21:07.695142 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d6178685e44ddc9e93adc34592078a" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a"
I20260807 18:21:07.695358 803 raft_consensus.cc:3037] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.695369 989 raft_consensus.cc:3037] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:07.698536 989 raft_consensus.cc:2454] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 1.
I20260807 18:21:07.698644 803 raft_consensus.cc:2454] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 1.
I20260807 18:21:07.698987 520 leader_election.cc:302] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:07.699378 1086 raft_consensus.cc:2781] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:07.699921 1086 raft_consensus.cc:686] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [term 1 LEADER]: Becoming Leader. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Running, Role: LEADER
I20260807 18:21:07.700657 1086 consensus_queue.cc:237] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.705480 387 catalog_manager.cc:5495] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001481e32beb42e1be1020b7991d0a63 (127.29.156.193). New cstate: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:07.706624 1110 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: TxnStatusTablet state changed. Reason: New leader 001481e32beb42e1be1020b7991d0a63. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.706931 1110 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:07.707710 1112 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:21:07.708039 1112 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:21:07.731396 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.724849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.0.0.1:39586)
0807 18:21:07.725197 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.725215 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.725390 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.725925 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.725930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.725943 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:07.726180 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.726194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.727434 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.727443 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.728652 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.728671 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.729497 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.729503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.729740 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.729800 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.730720 (+ 920us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.730745 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.730843 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.731009 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:21:07.731369 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.724836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.29.156.193:43641)
0807 18:21:07.725029 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.725036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.725301 (+ 265us) server_negotiation.cc:407] Connection header received
0807 18:21:07.725637 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.725642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.725734 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.725900 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:07.726336 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.727296 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.728820 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.729311 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.729491 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.730036 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.730592 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.730774 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.730871 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.730965 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:21:07.739970 803 raft_consensus.cc:1261] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Refusing update from remote peer 001481e32beb42e1be1020b7991d0a63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:07.740149 989 raft_consensus.cc:1261] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Refusing update from remote peer 001481e32beb42e1be1020b7991d0a63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:07.741151 1110 consensus_queue.cc:1035] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:07.741604 1086 consensus_queue.cc:1035] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:07.749332 1084 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: TxnStatusTablet state changed. Reason: New leader 001481e32beb42e1be1020b7991d0a63. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.749739 1084 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.754323 1116 mvcc.cc:205] Tried to move back new op lower bound from 7315975650252279808 to 7315975650105864192. Current Snapshot: MvccSnapshot[applied={T|T < 7315975650252279808}]
I20260807 18:21:07.754514 1119 mvcc.cc:205] Tried to move back new op lower bound from 7315975650252279808 to 7315975650105864192. Current Snapshot: MvccSnapshot[applied={T|T < 7315975650252279808}]
I20260807 18:21:07.755700 1110 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.755982 1110 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:07.757562 1084 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.757877 1084 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 0039e197a64d402c81557d8bb736d61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.763480 1085 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: TxnStatusTablet state changed. Reason: New leader 001481e32beb42e1be1020b7991d0a63. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.763792 1085 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.766988 1110 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.767302 1110 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 001481e32beb42e1be1020b7991d0a63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:21:07.772187 1085 tablet_replica.cc:439] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } } }
I20260807 18:21:07.772516 1085 tablet_replica.cc:442] T 39d6178685e44ddc9e93adc34592078a P 00d647fb147141ad9a6f448f6c88aba7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:21:07.778903 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.770792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.29.156.194:43463)
0807 18:21:07.771739 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.771746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.771766 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:21:07.771887 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.771893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.772044 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.772234 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:07.772942 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.774197 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.776330 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.776911 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.777082 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.777606 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.778201 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.778380 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.778473 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.778580 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751}
I20260807 18:21:07.798364 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.791512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:37345 (local address 127.29.156.193:43641)
0807 18:21:07.791763 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.791771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.792493 (+ 722us) server_negotiation.cc:407] Connection header received
0807 18:21:07.792673 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.792677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.792745 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.792829 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:07.793457 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.794323 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.796427 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.796933 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.797047 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.797612 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.797739 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.797813 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.798049 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.798136 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:21:07.798552 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.791576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.29.156.194:37345)
0807 18:21:07.792350 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.792371 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.792535 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.792876 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.792882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.792895 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:07.793261 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.793272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.794472 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.794479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.796266 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.796283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.797128 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.797136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.797317 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.797395 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.797846 (+ 451us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.797860 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.797945 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.798116 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":366,"threads_started":1}
I20260807 18:21:07.809511 989 raft_consensus.cc:1261] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Refusing update from remote peer 0039e197a64d402c81557d8bb736d61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:07.809706 619 raft_consensus.cc:1261] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Refusing update from remote peer 0039e197a64d402c81557d8bb736d61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:07.810807 1084 consensus_queue.cc:1035] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:07.811208 1091 consensus_queue.cc:1035] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:07.855180 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.847592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:43147 (local address 127.29.156.194:43463)
0807 18:21:07.847831 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:07.847839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:07.848146 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:21:07.848381 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:07.848387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:07.848467 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:07.848577 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:07.849213 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.850281 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.852926 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.853704 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.853856 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.854327 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.854472 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.854574 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:07.854742 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:21:07.854838 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:21:07.855329 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:07.846974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43463 (local address 127.29.156.193:43147)
0807 18:21:07.848034 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:07.848054 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:07.848236 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:07.848600 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:07.848604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:07.848622 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:07.849013 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.849022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.850448 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.850453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.852723 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:07.852753 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:07.853910 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:07.853914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:07.854052 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:07.854122 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:07.854597 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:07.854613 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:21:07.854713 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:21:07.854850 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":668,"threads_started":1}
I20260807 18:21:07.897500 388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:07.897823 388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:07.902532 388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Sending DeleteTablet for 3 replicas of tablet ad7853d3cbe144eb9a2cbae52b9b2163
I20260807 18:21:07.903988 962 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:07 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:58280
I20260807 18:21:07.904029 779 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:07 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:43420
I20260807 18:21:07.905277 589 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7853d3cbe144eb9a2cbae52b9b2163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:07 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39572
I20260807 18:21:07.906599 387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:07.907362 387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:07.919100 588 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (DEFAULT_TABLE table=test-table [id=40722092d33248ae97143a9b385f5c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.919113 961 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (DEFAULT_TABLE table=test-table [id=40722092d33248ae97143a9b385f5c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.919870 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9308e7d3c24c9d8455dc33e989e8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.920132 588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9308e7d3c24c9d8455dc33e989e8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.932062 1149 tablet_replica.cc:330] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: stopping tablet replica
I20260807 18:21:07.932700 1149 raft_consensus.cc:2229] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:07.933713 1149 raft_consensus.cc:2258] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:07.933928 778 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (DEFAULT_TABLE table=test-table [id=40722092d33248ae97143a9b385f5c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:07.934548 778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9308e7d3c24c9d8455dc33e989e8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:07.939124 1149 ts_tablet_manager.cc:1905] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:07.944308 1079 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Bootstrap starting.
I20260807 18:21:07.946673 1079 tablet_bootstrap.cc:654] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.949219 1079 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: No bootstrap required, opened a new log
I20260807 18:21:07.949422 1079 ts_tablet_manager.cc:1397] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 18:21:07.950551 1079 raft_consensus.cc:348] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.950928 1079 raft_consensus.cc:374] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.951030 1079 raft_consensus.cc:729] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.951351 1079 consensus_queue.cc:260] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.951581 1078 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Bootstrap starting.
I20260807 18:21:07.952092 1079 ts_tablet_manager.cc:1428] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:07.953662 1078 tablet_bootstrap.cc:654] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.962728 1077 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Bootstrap starting.
I20260807 18:21:07.965061 1077 tablet_bootstrap.cc:654] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:07.965533 1150 tablet_replica.cc:330] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: stopping tablet replica
I20260807 18:21:07.966689 1148 tablet_replica.cc:330] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: stopping tablet replica
I20260807 18:21:07.967166 1148 raft_consensus.cc:2229] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:07.967621 1148 raft_consensus.cc:2258] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:07.968075 1150 raft_consensus.cc:2229] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:07.968441 1078 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: No bootstrap required, opened a new log
I20260807 18:21:07.968732 1078 ts_tablet_manager.cc:1397] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.009s
I20260807 18:21:07.968482 1150 raft_consensus.cc:2258] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:07.969828 1078 raft_consensus.cc:348] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.970198 1078 raft_consensus.cc:374] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.970366 1078 raft_consensus.cc:729] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.970777 1078 consensus_queue.cc:260] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.971124 1150 ts_tablet_manager.cc:1905] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:07.971767 1078 ts_tablet_manager.cc:1428] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:07.972817 1149 ts_tablet_manager.cc:1918] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:07.973057 1149 log.cc:1196] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/ad7853d3cbe144eb9a2cbae52b9b2163
I20260807 18:21:07.973676 1149 ts_tablet_manager.cc:1939] T ad7853d3cbe144eb9a2cbae52b9b2163 P 0039e197a64d402c81557d8bb736d61a: Deleting consensus metadata
I20260807 18:21:07.980753 1077 tablet_bootstrap.cc:492] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: No bootstrap required, opened a new log
I20260807 18:21:07.981019 1077 ts_tablet_manager.cc:1397] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260807 18:21:07.981078 369 catalog_manager.cc:4841] TS 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463): tablet ad7853d3cbe144eb9a2cbae52b9b2163 (table test-table [id=33e1edb1ebee45a69a6d37719da46751]) successfully deleted
I20260807 18:21:07.981968 1077 raft_consensus.cc:348] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.982266 1077 raft_consensus.cc:374] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:07.982367 1077 raft_consensus.cc:729] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Initialized, Role: FOLLOWER
I20260807 18:21:07.982635 1077 consensus_queue.cc:260] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:07.983263 1077 ts_tablet_manager.cc:1428] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 18:21:07.985612 1148 ts_tablet_manager.cc:1905] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:07.992388 1150 ts_tablet_manager.cc:1918] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:07.992728 1150 log.cc:1196] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/ad7853d3cbe144eb9a2cbae52b9b2163
I20260807 18:21:07.993477 1150 ts_tablet_manager.cc:1939] T ad7853d3cbe144eb9a2cbae52b9b2163 P 001481e32beb42e1be1020b7991d0a63: Deleting consensus metadata
I20260807 18:21:07.994081 1148 ts_tablet_manager.cc:1918] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:07.994284 1148 log.cc:1196] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/ad7853d3cbe144eb9a2cbae52b9b2163
I20260807 18:21:07.994864 1148 ts_tablet_manager.cc:1939] T ad7853d3cbe144eb9a2cbae52b9b2163 P 00d647fb147141ad9a6f448f6c88aba7: Deleting consensus metadata
I20260807 18:21:07.995925 370 catalog_manager.cc:4841] TS 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641): tablet ad7853d3cbe144eb9a2cbae52b9b2163 (table test-table [id=33e1edb1ebee45a69a6d37719da46751]) successfully deleted
I20260807 18:21:07.996785 370 catalog_manager.cc:4841] TS 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535): tablet ad7853d3cbe144eb9a2cbae52b9b2163 (table test-table [id=33e1edb1ebee45a69a6d37719da46751]) successfully deleted
I20260807 18:21:08.040125 1110 raft_consensus.cc:482] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:08.040381 1110 raft_consensus.cc:504] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:08.041204 1110 leader_election.cc:288] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:08.041941 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9308e7d3c24c9d8455dc33e989e8c8" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7" is_pre_election: true
I20260807 18:21:08.041980 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9308e7d3c24c9d8455dc33e989e8c8" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a" is_pre_election: true
I20260807 18:21:08.042205 989 raft_consensus.cc:2454] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 0.
I20260807 18:21:08.042341 803 raft_consensus.cc:2454] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 0.
I20260807 18:21:08.043179 520 leader_election.cc:302] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:08.043581 1110 raft_consensus.cc:2781] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:08.043695 1110 raft_consensus.cc:482] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:08.043819 1110 raft_consensus.cc:3037] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:08.047241 1110 raft_consensus.cc:504] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:08.048504 989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9308e7d3c24c9d8455dc33e989e8c8" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7"
I20260807 18:21:08.048542 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9308e7d3c24c9d8455dc33e989e8c8" candidate_uuid: "001481e32beb42e1be1020b7991d0a63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a"
I20260807 18:21:08.048731 989 raft_consensus.cc:3037] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:08.048780 803 raft_consensus.cc:3037] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:08.051826 989 raft_consensus.cc:2454] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 1.
I20260807 18:21:08.052340 803 raft_consensus.cc:2454] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001481e32beb42e1be1020b7991d0a63 in term 1.
I20260807 18:21:08.052325 520 leader_election.cc:302] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:08.052695 1133 raft_consensus.cc:2781] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:08.052897 1133 raft_consensus.cc:686] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 1 LEADER]: Becoming Leader. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Running, Role: LEADER
I20260807 18:21:08.053720 1110 leader_election.cc:288] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [CANDIDATE]: Term 1 election: Requested vote from peers 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:08.053251 1133 consensus_queue.cc:237] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:08.057017 388 catalog_manager.cc:5495] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001481e32beb42e1be1020b7991d0a63 (127.29.156.193). New cstate: current_term: 1 leader_uuid: "001481e32beb42e1be1020b7991d0a63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:08.087973 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:08.080770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.29.156.193:43641)
0807 18:21:08.080963 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:08.080969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:08.080990 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:21:08.081059 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:08.081064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:08.081156 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:08.081251 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:08.081847 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.083106 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.085107 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.085847 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.085976 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:08.086590 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:08.086749 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:08.086831 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:08.087462 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 18:21:08.087608 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:21:08.106024 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:08.098758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:56639 (local address 127.29.156.193:43641)
0807 18:21:08.098954 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:08.098960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:08.099215 (+ 255us) server_negotiation.cc:407] Connection header received
0807 18:21:08.099454 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:08.099459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:08.099604 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:08.099697 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:08.100171 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.101379 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.103780 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.104504 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.104638 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:08.105127 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:08.105277 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:08.105374 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:08.105547 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 18:21:08.105639 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":67}
I20260807 18:21:08.106045 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:08.098721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.29.156.193:56639)
0807 18:21:08.099093 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:08.099121 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:08.099285 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:08.099743 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:08.099749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:08.099764 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:08.100007 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:08.100020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.101558 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.101564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:08.103620 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:08.103637 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:08.104704 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:08.104711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:08.104903 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:08.104956 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:08.105398 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:08.105416 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:21:08.105529 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:21:08.105689 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260807 18:21:08.118782 989 raft_consensus.cc:1261] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Refusing update from remote peer 001481e32beb42e1be1020b7991d0a63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:08.118932 803 raft_consensus.cc:1261] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Refusing update from remote peer 001481e32beb42e1be1020b7991d0a63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:08.119490 1086 consensus_queue.cc:1035] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:08.119822 1133 consensus_queue.cc:1035] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:21:08.171653 1116 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:21:09.797858 388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:09.798245 388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:09.803364 388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Sending DeleteTablet for 3 replicas of tablet 9c9308e7d3c24c9d8455dc33e989e8c8
I20260807 18:21:09.804636 778 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:09 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:43420
I20260807 18:21:09.804651 589 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:09 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39572
I20260807 18:21:09.804726 961 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9308e7d3c24c9d8455dc33e989e8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:09 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:58280
I20260807 18:21:09.817219 1189 tablet_replica.cc:330] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: stopping tablet replica
I20260807 18:21:09.817817 1189 raft_consensus.cc:2229] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:09.817987 1191 tablet_replica.cc:330] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: stopping tablet replica
I20260807 18:21:09.818231 1189 raft_consensus.cc:2258] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:09.818370 1191 raft_consensus.cc:2229] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:09.819038 1191 raft_consensus.cc:2258] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:09.820005 1189 ts_tablet_manager.cc:1905] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:09.820652 1191 ts_tablet_manager.cc:1905] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:09.826489 1188 tablet_replica.cc:330] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: stopping tablet replica
I20260807 18:21:09.826937 1188 raft_consensus.cc:2229] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:09.827893 1188 raft_consensus.cc:2258] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:09.829104 1188 ts_tablet_manager.cc:1905] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:09.830240 1189 ts_tablet_manager.cc:1918] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:09.830443 1189 log.cc:1196] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/9c9308e7d3c24c9d8455dc33e989e8c8
I20260807 18:21:09.830747 1191 ts_tablet_manager.cc:1918] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:09.831017 1189 ts_tablet_manager.cc:1939] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 0039e197a64d402c81557d8bb736d61a: Deleting consensus metadata
I20260807 18:21:09.831024 1191 log.cc:1196] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/9c9308e7d3c24c9d8455dc33e989e8c8
I20260807 18:21:09.831838 1191 ts_tablet_manager.cc:1939] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 00d647fb147141ad9a6f448f6c88aba7: Deleting consensus metadata
I20260807 18:21:09.832576 369 catalog_manager.cc:4841] TS 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463): tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (table test-table [id=40722092d33248ae97143a9b385f5c75]) successfully deleted
I20260807 18:21:09.833238 370 catalog_manager.cc:4841] TS 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535): tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (table test-table [id=40722092d33248ae97143a9b385f5c75]) successfully deleted
I20260807 18:21:09.837883 1188 ts_tablet_manager.cc:1918] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:09.838076 1188 log.cc:1196] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/9c9308e7d3c24c9d8455dc33e989e8c8
I20260807 18:21:09.838717 1188 ts_tablet_manager.cc:1939] T 9c9308e7d3c24c9d8455dc33e989e8c8 P 001481e32beb42e1be1020b7991d0a63: Deleting consensus metadata
I20260807 18:21:09.840217 370 catalog_manager.cc:4841] TS 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641): tablet 9c9308e7d3c24c9d8455dc33e989e8c8 (table test-table [id=40722092d33248ae97143a9b385f5c75]) successfully deleted
Aug 07 18:21:09 dist-test-slave-sb2z krb5kdc[325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126869, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:21:09 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126869, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:09.880388 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:09.863270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.29.156.254:43477)
0807 18:21:09.863957 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:09.863962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:09.864100 (+ 138us) server_negotiation.cc:407] Connection header received
0807 18:21:09.864269 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:09.864273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:09.864340 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:09.864446 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:09.866580 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:09.867382 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:09.868834 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:09.869223 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:09.869319 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:09.874411 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:09.874416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:09.874419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:09.874457 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:09.877563 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:09.878290 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:09.878294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:09.878296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:09.878450 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:09.878831 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:09.878834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:09.878836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:09.879144 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:09.879352 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:09.879861 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:21:09.880073 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":400,"threads_started":1}
I20260807 18:21:09.884812 388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:09.885615 388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:09.892843 589 tablet_service.cc:1467] Processing CreateTablet for tablet db3cad8142924433bade7bfdda23dbfb (DEFAULT_TABLE table=test-table [id=439bb082d0184647a76142d407f0da2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:09.893080 961 tablet_service.cc:1467] Processing CreateTablet for tablet db3cad8142924433bade7bfdda23dbfb (DEFAULT_TABLE table=test-table [id=439bb082d0184647a76142d407f0da2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:09.893080 778 tablet_service.cc:1467] Processing CreateTablet for tablet db3cad8142924433bade7bfdda23dbfb (DEFAULT_TABLE table=test-table [id=439bb082d0184647a76142d407f0da2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:09.893373 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3cad8142924433bade7bfdda23dbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:09.893617 778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3cad8142924433bade7bfdda23dbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:09.893651 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3cad8142924433bade7bfdda23dbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:09.900204 1203 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Bootstrap starting.
I20260807 18:21:09.901238 1205 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Bootstrap starting.
I20260807 18:21:09.902223 1204 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Bootstrap starting.
I20260807 18:21:09.903323 1203 tablet_bootstrap.cc:654] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:09.903774 1205 tablet_bootstrap.cc:654] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:09.904553 1204 tablet_bootstrap.cc:654] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:09.906594 1205 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: No bootstrap required, opened a new log
I20260807 18:21:09.906730 1203 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: No bootstrap required, opened a new log
I20260807 18:21:09.906780 1205 ts_tablet_manager.cc:1397] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 18:21:09.906984 1203 ts_tablet_manager.cc:1397] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:21:09.908180 1204 tablet_bootstrap.cc:492] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: No bootstrap required, opened a new log
I20260807 18:21:09.908058 1203 raft_consensus.cc:348] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.908092 1205 raft_consensus.cc:348] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.908370 1204 ts_tablet_manager.cc:1397] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:21:09.908366 1203 raft_consensus.cc:374] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:09.908493 1203 raft_consensus.cc:729] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Initialized, Role: FOLLOWER
I20260807 18:21:09.908541 1205 raft_consensus.cc:374] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:09.908749 1205 raft_consensus.cc:729] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Initialized, Role: FOLLOWER
I20260807 18:21:09.908802 1203 consensus_queue.cc:260] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.909078 1205 consensus_queue.cc:260] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.909575 1203 ts_tablet_manager.cc:1428] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 18:21:09.909400 1204 raft_consensus.cc:348] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.909806 1204 raft_consensus.cc:374] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:09.909950 1204 raft_consensus.cc:729] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Initialized, Role: FOLLOWER
I20260807 18:21:09.910373 1204 consensus_queue.cc:260] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:09.910604 1205 ts_tablet_manager.cc:1428] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:09.912441 1204 ts_tablet_manager.cc:1428] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:21:10.044157 1210 raft_consensus.cc:482] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:10.044487 1210 raft_consensus.cc:504] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.046355 1210 leader_election.cc:288] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463)
I20260807 18:21:10.057296 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.046660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:42311 (local address 127.29.156.193:43641)
0807 18:21:10.048811 (+ 2151us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.048818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.048847 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:21:10.048953 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.048958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.049042 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.049191 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:10.050243 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.051331 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.054337 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.055838 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.056090 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.056176 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.056617 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.056726 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1969,"thread_start_us":1734,"threads_started":1}
I20260807 18:21:10.057488 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.047084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33209 (local address 127.29.156.194:43463)
0807 18:21:10.048460 (+ 1376us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.048467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.049270 (+ 803us) server_negotiation.cc:407] Connection header received
0807 18:21:10.049579 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.049585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.049689 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.049847 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:10.050783 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.052068 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.054650 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.056204 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.056380 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.056433 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.056831 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.057017 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":959,"threads_started":1}
I20260807 18:21:10.057624 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.046654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43463 (local address 127.29.156.195:33209)
0807 18:21:10.049150 (+ 2496us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:10.049170 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:10.049370 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:10.049868 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:10.049871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:10.049891 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:10.050558 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.050576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.052231 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.052239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.054475 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.054485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.056474 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.056481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.056661 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.056691 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:21:10.056773 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:21:10.056977 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2297,"mutex_wait_us":47,"spinlock_wait_cycles":3840,"thread_start_us":2064,"threads_started":1}
I20260807 18:21:10.057673 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.046569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.29.156.195:42311)
0807 18:21:10.047457 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:10.047486 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:10.047700 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:10.049252 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:10.049257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:10.049379 (+ 122us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:10.049929 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.049945 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.051566 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.051575 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.054139 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.054154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.056121 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.056126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.056378 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.056456 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 18:21:10.056565 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:21:10.056705 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":25,"spinlock_wait_cycles":73088,"thread_start_us":507,"threads_started":1}
I20260807 18:21:10.058645 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3cad8142924433bade7bfdda23dbfb" candidate_uuid: "00d647fb147141ad9a6f448f6c88aba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63" is_pre_election: true
I20260807 18:21:10.058645 802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3cad8142924433bade7bfdda23dbfb" candidate_uuid: "00d647fb147141ad9a6f448f6c88aba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a" is_pre_election: true
I20260807 18:21:10.058928 802 raft_consensus.cc:2454] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d647fb147141ad9a6f448f6c88aba7 in term 0.
I20260807 18:21:10.058945 619 raft_consensus.cc:2454] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d647fb147141ad9a6f448f6c88aba7 in term 0.
I20260807 18:21:10.059626 904 leader_election.cc:302] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 0039e197a64d402c81557d8bb736d61a, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:10.060189 1210 raft_consensus.cc:2781] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:10.060400 1210 raft_consensus.cc:482] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:10.060570 1210 raft_consensus.cc:3037] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.063689 1210 raft_consensus.cc:504] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.064499 1210 leader_election.cc:288] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [CANDIDATE]: Term 1 election: Requested vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463)
I20260807 18:21:10.065060 802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3cad8142924433bade7bfdda23dbfb" candidate_uuid: "00d647fb147141ad9a6f448f6c88aba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0039e197a64d402c81557d8bb736d61a"
I20260807 18:21:10.065169 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3cad8142924433bade7bfdda23dbfb" candidate_uuid: "00d647fb147141ad9a6f448f6c88aba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63"
I20260807 18:21:10.065310 802 raft_consensus.cc:3037] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.065487 619 raft_consensus.cc:3037] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.067999 802 raft_consensus.cc:2454] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d647fb147141ad9a6f448f6c88aba7 in term 1.
I20260807 18:21:10.068387 619 raft_consensus.cc:2454] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d647fb147141ad9a6f448f6c88aba7 in term 1.
I20260807 18:21:10.068487 900 leader_election.cc:302] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0039e197a64d402c81557d8bb736d61a, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:10.068877 1210 raft_consensus.cc:2781] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:10.069599 1210 raft_consensus.cc:686] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 1 LEADER]: Becoming Leader. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Running, Role: LEADER
I20260807 18:21:10.070104 1210 consensus_queue.cc:237] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.073947 388 catalog_manager.cc:5495] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "00d647fb147141ad9a6f448f6c88aba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } health_report { overall_health: HEALTHY } } }
I20260807 18:21:10.114677 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.107488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.29.156.195:41535)
0807 18:21:10.108469 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.108473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.108494 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:21:10.108584 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.108588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.108678 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.108828 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:10.109235 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.110187 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.111686 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.112266 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.112367 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.113090 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.113742 (+ 652us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.113866 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.114151 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.114289 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":647,"threads_started":1}
I20260807 18:21:10.135306 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.127751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:47123 (local address 127.29.156.193:43641)
0807 18:21:10.127984 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.127989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.129146 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 18:21:10.129462 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.129469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.129551 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.129643 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:10.130358 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.131195 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.133294 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.133868 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.133969 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.134637 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.134760 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.134840 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.135024 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.135094 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:21:10.135707 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.127644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43641 (local address 127.29.156.195:47123)
0807 18:21:10.128954 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:10.128989 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:10.129296 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:10.129713 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:10.129716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:10.129727 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:10.130185 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.130196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.131433 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.131438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.133144 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.133158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.134144 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.134150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.134368 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.134430 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.134873 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.134884 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:21:10.134988 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:21:10.135160 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":635,"threads_started":1}
I20260807 18:21:10.150882 619 raft_consensus.cc:1261] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Refusing update from remote peer 00d647fb147141ad9a6f448f6c88aba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:10.150914 802 raft_consensus.cc:1261] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Refusing update from remote peer 00d647fb147141ad9a6f448f6c88aba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:10.151870 1210 consensus_queue.cc:1035] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:10.152262 1215 consensus_queue.cc:1035] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:10.163774 1169 mvcc.cc:205] Tried to move back new op lower bound from 7315975660130906112 to 7315975659811065856. Current Snapshot: MvccSnapshot[applied={T|T < 7315975660130906112}]
I20260807 18:21:10.199208 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.183294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:51027 (local address 127.29.156.195:41535)
0807 18:21:10.183514 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.183544 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.192647 (+ 9103us) server_negotiation.cc:407] Connection header received
0807 18:21:10.192987 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.192994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.193083 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.193227 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:10.194041 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.194871 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.196874 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.197535 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.197733 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.198353 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.198477 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.198598 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.198770 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.198857 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:21:10.199478 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.182935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:41535 (local address 127.29.156.193:51027)
0807 18:21:10.192512 (+ 9577us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:10.192536 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:10.192778 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:10.193540 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:10.193545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:10.193560 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:21:10.193841 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.193849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.195046 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.195053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.196695 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:10.196709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.197748 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.197754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:10.198068 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.198116 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.198601 (+ 485us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.198617 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:21:10.198741 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:21:10.198895 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9396,"thread_start_us":9222,"threads_started":1}
I20260807 18:21:10.233269 387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:10.233609 387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:10.237474 387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Sending DeleteTablet for 3 replicas of tablet db3cad8142924433bade7bfdda23dbfb
I20260807 18:21:10.238408 961 tablet_service.cc:1514] Processing DeleteTablet for tablet db3cad8142924433bade7bfdda23dbfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:58280
I20260807 18:21:10.238454 778 tablet_service.cc:1514] Processing DeleteTablet for tablet db3cad8142924433bade7bfdda23dbfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:43420
I20260807 18:21:10.238734 1191 tablet_replica.cc:330] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: stopping tablet replica
I20260807 18:21:10.238793 1189 tablet_replica.cc:330] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: stopping tablet replica
I20260807 18:21:10.238960 1191 raft_consensus.cc:2229] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:10.239118 1189 raft_consensus.cc:2229] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:10.239346 1189 raft_consensus.cc:2258] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:10.239557 1191 raft_consensus.cc:2258] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:10.239912 387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:10.240633 387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:10.240955 1191 ts_tablet_manager.cc:1905] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:10.242631 1189 ts_tablet_manager.cc:1905] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:10.247771 1189 ts_tablet_manager.cc:1918] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:10.247951 1189 log.cc:1196] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/db3cad8142924433bade7bfdda23dbfb
I20260807 18:21:10.248603 1189 ts_tablet_manager.cc:1939] T db3cad8142924433bade7bfdda23dbfb P 0039e197a64d402c81557d8bb736d61a: Deleting consensus metadata
I20260807 18:21:10.248695 1191 ts_tablet_manager.cc:1918] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:10.248862 1191 log.cc:1196] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/db3cad8142924433bade7bfdda23dbfb
I20260807 18:21:10.249486 1191 ts_tablet_manager.cc:1939] T db3cad8142924433bade7bfdda23dbfb P 00d647fb147141ad9a6f448f6c88aba7: Deleting consensus metadata
I20260807 18:21:10.249887 369 catalog_manager.cc:4841] TS 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463): tablet db3cad8142924433bade7bfdda23dbfb (table test-table [id=439bb082d0184647a76142d407f0da2d]) successfully deleted
I20260807 18:21:10.250861 370 catalog_manager.cc:4841] TS 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535): tablet db3cad8142924433bade7bfdda23dbfb (table test-table [id=439bb082d0184647a76142d407f0da2d]) successfully deleted
I20260807 18:21:10.251503 589 tablet_service.cc:1514] Processing DeleteTablet for tablet db3cad8142924433bade7bfdda23dbfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39572
I20260807 18:21:10.251875 1188 tablet_replica.cc:330] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: stopping tablet replica
I20260807 18:21:10.252099 1188 raft_consensus.cc:2229] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:10.252305 1188 raft_consensus.cc:2258] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:10.253188 589 tablet_service.cc:1467] Processing CreateTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b (DEFAULT_TABLE table=test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:10.253665 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 090a60dfca1a4d16a7c4b649e4b9d68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:10.253650 778 tablet_service.cc:1467] Processing CreateTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b (DEFAULT_TABLE table=test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:10.253835 1188 ts_tablet_manager.cc:1905] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:10.254169 778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 090a60dfca1a4d16a7c4b649e4b9d68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:10.254884 961 tablet_service.cc:1467] Processing CreateTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b (DEFAULT_TABLE table=test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:10.255277 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 090a60dfca1a4d16a7c4b649e4b9d68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:10.261518 1205 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Bootstrap starting.
I20260807 18:21:10.262252 1204 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Bootstrap starting.
I20260807 18:21:10.263715 1205 tablet_bootstrap.cc:654] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:10.264391 1204 tablet_bootstrap.cc:654] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:10.265385 1203 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Bootstrap starting.
I20260807 18:21:10.267495 1203 tablet_bootstrap.cc:654] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:10.268639 1188 ts_tablet_manager.cc:1918] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:21:10.269588 1188 log.cc:1196] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/db3cad8142924433bade7bfdda23dbfb
I20260807 18:21:10.270435 1188 ts_tablet_manager.cc:1939] T db3cad8142924433bade7bfdda23dbfb P 001481e32beb42e1be1020b7991d0a63: Deleting consensus metadata
I20260807 18:21:10.272341 370 catalog_manager.cc:4841] TS 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641): tablet db3cad8142924433bade7bfdda23dbfb (table test-table [id=439bb082d0184647a76142d407f0da2d]) successfully deleted
I20260807 18:21:10.275759 1205 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: No bootstrap required, opened a new log
I20260807 18:21:10.276085 1205 ts_tablet_manager.cc:1397] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260807 18:21:10.277281 1205 raft_consensus.cc:348] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.277500 1203 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: No bootstrap required, opened a new log
I20260807 18:21:10.277724 1203 ts_tablet_manager.cc:1397] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20260807 18:21:10.277751 1205 raft_consensus.cc:374] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:10.277951 1205 raft_consensus.cc:729] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Initialized, Role: FOLLOWER
I20260807 18:21:10.278329 1205 consensus_queue.cc:260] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.278914 1203 raft_consensus.cc:348] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.279241 1203 raft_consensus.cc:374] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:10.279335 1203 raft_consensus.cc:729] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001481e32beb42e1be1020b7991d0a63, State: Initialized, Role: FOLLOWER
I20260807 18:21:10.279798 1203 consensus_queue.cc:260] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.280532 1203 ts_tablet_manager.cc:1428] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 18:21:10.281453 1205 ts_tablet_manager.cc:1428] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:21:10.283692 1204 tablet_bootstrap.cc:492] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: No bootstrap required, opened a new log
I20260807 18:21:10.283944 1204 ts_tablet_manager.cc:1397] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.011s
I20260807 18:21:10.284937 1204 raft_consensus.cc:348] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.285323 1204 raft_consensus.cc:374] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:10.285521 1204 raft_consensus.cc:729] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d647fb147141ad9a6f448f6c88aba7, State: Initialized, Role: FOLLOWER
I20260807 18:21:10.285876 1204 consensus_queue.cc:260] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.286618 1204 ts_tablet_manager.cc:1428] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:21:10.351045 1209 raft_consensus.cc:482] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:10.351365 1209 raft_consensus.cc:504] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.352398 1209 leader_election.cc:288] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:10.353286 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "090a60dfca1a4d16a7c4b649e4b9d68b" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7" is_pre_election: true
I20260807 18:21:10.353358 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "090a60dfca1a4d16a7c4b649e4b9d68b" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63" is_pre_election: true
I20260807 18:21:10.353598 986 raft_consensus.cc:2454] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 0.
I20260807 18:21:10.353659 619 raft_consensus.cc:2454] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 0.
I20260807 18:21:10.354197 720 leader_election.cc:302] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0039e197a64d402c81557d8bb736d61a, 00d647fb147141ad9a6f448f6c88aba7; no voters:
I20260807 18:21:10.354575 1209 raft_consensus.cc:2781] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:10.354722 1209 raft_consensus.cc:482] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:10.354926 1209 raft_consensus.cc:3037] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.357944 1209 raft_consensus.cc:504] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.358743 1209 leader_election.cc:288] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 election: Requested vote from peers 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641), 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535)
I20260807 18:21:10.359282 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "090a60dfca1a4d16a7c4b649e4b9d68b" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001481e32beb42e1be1020b7991d0a63"
I20260807 18:21:10.359494 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "090a60dfca1a4d16a7c4b649e4b9d68b" candidate_uuid: "0039e197a64d402c81557d8bb736d61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d647fb147141ad9a6f448f6c88aba7"
I20260807 18:21:10.359678 619 raft_consensus.cc:3037] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.359827 986 raft_consensus.cc:3037] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:10.363112 986 raft_consensus.cc:2454] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 1.
I20260807 18:21:10.363132 619 raft_consensus.cc:2454] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0039e197a64d402c81557d8bb736d61a in term 1.
I20260807 18:21:10.363602 720 leader_election.cc:302] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001481e32beb42e1be1020b7991d0a63, 0039e197a64d402c81557d8bb736d61a; no voters:
I20260807 18:21:10.364086 1209 raft_consensus.cc:2781] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:10.364645 1209 raft_consensus.cc:686] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 1 LEADER]: Becoming Leader. State: Replica: 0039e197a64d402c81557d8bb736d61a, State: Running, Role: LEADER
I20260807 18:21:10.364965 1209 consensus_queue.cc:237] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } }
I20260807 18:21:10.368417 388 catalog_manager.cc:5495] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0039e197a64d402c81557d8bb736d61a (127.29.156.194). New cstate: current_term: 1 leader_uuid: "0039e197a64d402c81557d8bb736d61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0039e197a64d402c81557d8bb736d61a" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:10.412233 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:10.405544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.29.156.194:43463)
0807 18:21:10.405788 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:10.405795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:10.406139 (+ 344us) server_negotiation.cc:407] Connection header received
0807 18:21:10.406338 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:10.406342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:10.406429 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:10.406505 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:10.407034 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.407881 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.410062 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:10.410676 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:10.410795 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:10.411406 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:10.411518 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:10.411663 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:10.411846 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 18:21:10.411912 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:21:10.426210 986 raft_consensus.cc:1261] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Refusing update from remote peer 0039e197a64d402c81557d8bb736d61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:10.426290 619 raft_consensus.cc:1261] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Refusing update from remote peer 0039e197a64d402c81557d8bb736d61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:10.426932 1239 consensus_queue.cc:1035] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001481e32beb42e1be1020b7991d0a63" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:10.427491 1209 consensus_queue.cc:1035] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d647fb147141ad9a6f448f6c88aba7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:21:10.462453 1170 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:21:12.067229 388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:12.067623 388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:12.072499 388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5402b5c42a3344a993ba185350f85b87: Sending DeleteTablet for 3 replicas of tablet 090a60dfca1a4d16a7c4b649e4b9d68b
I20260807 18:21:12.073453 589 tablet_service.cc:1514] Processing DeleteTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:12 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39572
I20260807 18:21:12.073643 961 tablet_service.cc:1514] Processing DeleteTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:12 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:58280
I20260807 18:21:12.073681 778 tablet_service.cc:1514] Processing DeleteTablet for tablet 090a60dfca1a4d16a7c4b649e4b9d68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:12 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:43420
I20260807 18:21:12.077874 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 495
I20260807 18:21:12.088125 1278 tablet_replica.cc:330] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: stopping tablet replica
I20260807 18:21:12.088512 1278 raft_consensus.cc:2229] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:12.089213 1278 raft_consensus.cc:2258] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:12.089473 1277 tablet_replica.cc:330] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: stopping tablet replica
I20260807 18:21:12.089819 1277 raft_consensus.cc:2229] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:12.090212 1280 tablet_replica.cc:330] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: stopping tablet replica
I20260807 18:21:12.090194 1277 raft_consensus.cc:2258] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:12.090445 1280 raft_consensus.cc:2229] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:12.090729 1280 raft_consensus.cc:2258] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:12.091383 1278 ts_tablet_manager.cc:1905] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:12.091696 1277 ts_tablet_manager.cc:1905] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:12.092602 1280 ts_tablet_manager.cc:1905] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:12.099030 1278 ts_tablet_manager.cc:1918] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:12.099231 1278 log.cc:1196] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/090a60dfca1a4d16a7c4b649e4b9d68b
I20260807 18:21:12.099509 1277 ts_tablet_manager.cc:1918] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:12.099757 1277 log.cc:1196] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/090a60dfca1a4d16a7c4b649e4b9d68b
I20260807 18:21:12.099789 1278 ts_tablet_manager.cc:1939] T 090a60dfca1a4d16a7c4b649e4b9d68b P 0039e197a64d402c81557d8bb736d61a: Deleting consensus metadata
I20260807 18:21:12.099833 1280 ts_tablet_manager.cc:1918] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:21:12.099995 1280 log.cc:1196] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/090a60dfca1a4d16a7c4b649e4b9d68b
I20260807 18:21:12.100611 1277 ts_tablet_manager.cc:1939] T 090a60dfca1a4d16a7c4b649e4b9d68b P 001481e32beb42e1be1020b7991d0a63: Deleting consensus metadata
I20260807 18:21:12.100623 1280 ts_tablet_manager.cc:1939] T 090a60dfca1a4d16a7c4b649e4b9d68b P 00d647fb147141ad9a6f448f6c88aba7: Deleting consensus metadata
I20260807 18:21:12.101181 369 catalog_manager.cc:4841] TS 0039e197a64d402c81557d8bb736d61a (127.29.156.194:43463): tablet 090a60dfca1a4d16a7c4b649e4b9d68b (table test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]) successfully deleted
I20260807 18:21:12.101763 370 catalog_manager.cc:4841] TS 001481e32beb42e1be1020b7991d0a63 (127.29.156.193:43641): tablet 090a60dfca1a4d16a7c4b649e4b9d68b (table test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]) successfully deleted
I20260807 18:21:12.101990 370 catalog_manager.cc:4841] TS 00d647fb147141ad9a6f448f6c88aba7 (127.29.156.195:41535): tablet 090a60dfca1a4d16a7c4b649e4b9d68b (table test-table [id=88f6fa0208a74c049b1a45e8c4dc2a81]) successfully deleted
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126869, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:12.116101 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:12.079607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.29.156.193:43641)
0807 18:21:12.091370 (+ 11763us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:12.091380 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:12.092324 (+ 944us) server_negotiation.cc:407] Connection header received
0807 18:21:12.092402 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:12.092408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:12.092552 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:12.092686 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:12.095074 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.096472 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.098633 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.099353 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.099468 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:12.108441 (+ 8973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:12.108467 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:12.108486 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:12.108536 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:12.112460 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.113316 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.113326 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.113328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.113530 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.113984 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.113988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.113990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.114486 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:12.114777 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:12.115410 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 18:21:12.115686 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11621,"thread_start_us":11964,"threads_started":1}
W20260807 18:21:12.117081 675 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35660
W20260807 18:21:12.117874 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 495: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:12.118139 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 495
I20260807 18:21:12.156982 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 688
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126869, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:12.175979 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:12.157814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54460 (local address 127.29.156.194:43463)
0807 18:21:12.158480 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:12.158484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:12.158501 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:21:12.158557 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:12.158561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:12.158618 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:12.158767 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:12.160392 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.161220 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.162583 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.163027 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.163177 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:12.168681 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:12.168712 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:12.168740 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:12.168784 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:12.172395 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.173284 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.173292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.173295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.173448 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.173941 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.173945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.173947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.174400 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:12.174659 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:12.175063 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:21:12.175287 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":375,"threads_started":1}
W20260807 18:21:12.176918 861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54460
W20260807 18:21:12.177820 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 688: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:12.178053 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 688
I20260807 18:21:12.215361 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 871
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126869, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:12.234304 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:12.216399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.29.156.195:41535)
0807 18:21:12.217016 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:12.217021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:12.217039 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:21:12.217098 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:12.217101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:12.217160 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:12.217257 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:12.218812 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.219715 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.221114 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:12.221563 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:12.221670 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:12.226706 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:12.226725 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:12.226743 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:12.226799 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:12.230609 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.231672 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.231682 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.231684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.231843 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:12.232491 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:12.232500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:12.232504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:12.233013 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:12.233314 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:12.233822 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 18:21:12.234031 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":355,"threads_started":1}
W20260807 18:21:12.235209 1041 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50250
W20260807 18:21:12.236012 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 871: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:12.236246 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 871
I20260807 18:21:12.266492 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 343
I20260807 18:21:12.268075 446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:12.394563 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 343
2026-08-07T18:21:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8988 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[1291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[1291](info): set up 2 sockets
Aug 07 18:21:12 dist-test-slave-sb2z krb5kdc[1291](info): commencing operation
krb5kdc: starting...
W20260807 18:21:13.793002 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:13 dist-test-slave-sb2z krb5kdc[1291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:14.039633 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40671
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:14.381742 1313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:14.382046 1313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:14.382241 1313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:14.389654 1313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:14.389772 1313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:14.389854 1313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:14.389961 1313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:14.390025 1313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:14.400058 1313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:14.401450 1313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:14.402845 1313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:14.415648 1324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:14.415700 1321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:14.415729 1322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:14.416735 1313 server_base.cc:1034] running on GCE node
I20260807 18:21:14.417650 1313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:14.419173 1313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:14.420410 1313 hybrid_clock.cc:648] HybridClock initialized: now 1786126874420350 us; error 42 us; skew 500 ppm
I20260807 18:21:14.420755 1313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:14 dist-test-slave-sb2z krb5kdc[1291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126874, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:14.424129 1313 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:14.425704 1313 webserver.cc:460] Webserver started at http://127.29.156.254:38089/ using document root <none> and password file <none>
I20260807 18:21:14.426374 1313 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:14.426477 1313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:14.426735 1313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:14.429797 1313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "1c939f4da5ee424b9668f2daedfb4c5a"
format_stamp: "Formatted at 2026-08-07 18:21:14 on dist-test-slave-sb2z"
server_key: "7e2ce528bbdacb0e0808249ed79ca223"
server_key_iv: "36f4d48b7443da985da862e444b3f52d"
server_key_version: "encryptionkey@0"
I20260807 18:21:14.430835 1313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "1c939f4da5ee424b9668f2daedfb4c5a"
format_stamp: "Formatted at 2026-08-07 18:21:14 on dist-test-slave-sb2z"
server_key: "7e2ce528bbdacb0e0808249ed79ca223"
server_key_iv: "36f4d48b7443da985da862e444b3f52d"
server_key_version: "encryptionkey@0"
I20260807 18:21:14.437119 1313 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:21:14.440958 1335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:14.442548 1313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:21:14.442776 1313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "1c939f4da5ee424b9668f2daedfb4c5a"
format_stamp: "Formatted at 2026-08-07 18:21:14 on dist-test-slave-sb2z"
server_key: "7e2ce528bbdacb0e0808249ed79ca223"
server_key_iv: "36f4d48b7443da985da862e444b3f52d"
server_key_version: "encryptionkey@0"
I20260807 18:21:14.443017 1313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:14.457180 1313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:14.462173 1313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:14.462496 1313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:14.482975 1313 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:40671
I20260807 18:21:14.483003 1398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:40671 every 8 connection(s)
I20260807 18:21:14.484529 1313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:14.485283 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 1313
I20260807 18:21:14.485574 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:14.486340 30323 external_mini_cluster.cc:1428] Setting key 5406cf0291f0e12422220eb4fdb68809
I20260807 18:21:14.489265 1400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:21:14 dist-test-slave-sb2z krb5kdc[1291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:14.503450 1400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a: Bootstrap starting.
I20260807 18:21:14.506822 1400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:14.508278 1400 log.cc:824] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:14.511327 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:14.488361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.29.156.254:40671)
0807 18:21:14.489190 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:14.489202 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:14.489246 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:21:14.490996 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:14.491018 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:14.491402 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:14.491823 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:14.493725 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:14.495118 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:14.496999 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:14.497712 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:14.497848 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:14.503139 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:14.503148 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:14.503165 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:14.503209 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:14.506367 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:14.507122 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:14.507132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:14.507140 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:14.507386 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:14.507913 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:14.507925 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:14.507930 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:14.508690 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:14.509062 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:14.509539 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 18:21:14.509948 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":401,"threads_started":1}
I20260807 18:21:14.512120 1400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a: No bootstrap required, opened a new log
I20260807 18:21:14.520164 1400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } }
I20260807 18:21:14.520517 1400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:14.520596 1400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c939f4da5ee424b9668f2daedfb4c5a, State: Initialized, Role: FOLLOWER
I20260807 18:21:14.521102 1400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } }
I20260807 18:21:14.521313 1400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:14.521411 1400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:14.521538 1400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:14.523697 1400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } }
I20260807 18:21:14.524169 1400 leader_election.cc:302] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c939f4da5ee424b9668f2daedfb4c5a; no voters:
I20260807 18:21:14.524863 1400 leader_election.cc:288] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:14.525135 1405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:14.525880 1405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [term 1 LEADER]: Becoming Leader. State: Replica: 1c939f4da5ee424b9668f2daedfb4c5a, State: Running, Role: LEADER
I20260807 18:21:14.526455 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } }
I20260807 18:21:14.527163 1400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:14.529393 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } } }
I20260807 18:21:14.529831 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:14.529788 1408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c939f4da5ee424b9668f2daedfb4c5a. Latest consensus state: current_term: 1 leader_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c939f4da5ee424b9668f2daedfb4c5a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40671 } } }
I20260807 18:21:14.530121 1408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:14.532379 1417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:14.537283 1417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:14.546903 1417 catalog_manager.cc:1347] Generated new cluster ID: 1538f6d9429844de8a098fb81c1d30d7
I20260807 18:21:14.547029 1417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:14.574150 1417 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:14.576049 1417 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:14.582774 1417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c939f4da5ee424b9668f2daedfb4c5a: Generated new TSK 0
I20260807 18:21:14.583675 1417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:14.651913 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:15.012884 1437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:15.013206 1437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:15.013381 1437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:15.021116 1437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:15.021243 1437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:15.021433 1437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:15.032227 1437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:15.033622 1437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:15.035005 1437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:15.048887 1442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:15.048925 1443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:15.048842 1445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:15.050190 1437 server_base.cc:1034] running on GCE node
I20260807 18:21:15.050839 1437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:15.051884 1437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:15.053153 1437 hybrid_clock.cc:648] HybridClock initialized: now 1786126875053108 us; error 39 us; skew 500 ppm
I20260807 18:21:15.053478 1437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:15 dist-test-slave-sb2z krb5kdc[1291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126875, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:15.056923 1437 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:15.058498 1437 webserver.cc:460] Webserver started at http://127.29.156.193:42729/ using document root <none> and password file <none>
I20260807 18:21:15.059221 1437 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:15.059353 1437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:15.059736 1437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:15.062752 1437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "39dabdcfcc9640e08d04ecb15713e4fe"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "b54d5e4032ea160b81620bab2f0b8f5f"
server_key_iv: "2c2c6dd49c5d9b113498fe483be9df3e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.063710 1437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "39dabdcfcc9640e08d04ecb15713e4fe"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "b54d5e4032ea160b81620bab2f0b8f5f"
server_key_iv: "2c2c6dd49c5d9b113498fe483be9df3e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.068982 1437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 18:21:15.072575 1454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.074047 1437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:15.074226 1437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "39dabdcfcc9640e08d04ecb15713e4fe"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "b54d5e4032ea160b81620bab2f0b8f5f"
server_key_iv: "2c2c6dd49c5d9b113498fe483be9df3e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.074425 1437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:15.121384 1437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:15.126339 1437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:15.126677 1437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:15.127915 1437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:15.129420 1437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:15.129534 1437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.129683 1437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:15.129767 1437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.166716 1437 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:39845
I20260807 18:21:15.166771 1607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:39845 every 8 connection(s)
I20260807 18:21:15.168062 1437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:15.168973 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 1437
I20260807 18:21:15.169214 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:15.169909 30323 external_mini_cluster.cc:1428] Setting key 9f67746a18c03c21ab4821810521a575
Aug 07 18:21:15 dist-test-slave-sb2z krb5kdc[1291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126875, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:15.191248 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:15.170926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:55519 (local address 127.29.156.254:40671)
0807 18:21:15.171455 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:15.171461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:15.172617 (+ 1156us) server_negotiation.cc:407] Connection header received
0807 18:21:15.174675 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:15.174683 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:15.174774 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:15.174900 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:15.176503 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.177686 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.180066 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.180788 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.180963 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:15.184930 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:15.184935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:15.184939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:15.184972 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:15.187763 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:15.188515 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:15.188520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:15.188522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:15.188647 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:15.189140 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:15.189145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:15.189147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:15.189420 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:15.189583 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:15.190448 (+ 865us) server_negotiation.cc:299] Negotiation successful
0807 18:21:15.190718 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260807 18:21:15.191905 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:15.171572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40671 (local address 127.29.156.193:55519)
0807 18:21:15.172416 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:15.172464 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:15.174387 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:15.175100 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:15.175107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:15.175755 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:15.176330 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:15.176347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.177847 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.177855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:15.179902 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:15.179920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.181121 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.181130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:15.181368 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:15.182124 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:15.182148 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:15.184679 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:15.187910 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:15.187920 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:15.187937 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:15.188376 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:15.188761 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:15.188766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:15.188770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:15.189002 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:15.189557 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:15.189573 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:15.189926 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 18:21:15.190249 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 18:21:15.190668 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":353,"threads_started":1}
I20260807 18:21:15.193602 1608 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40671
I20260807 18:21:15.193971 1608 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
I20260807 18:21:15.195053 1608 heartbeater.cc:511] Master 127.29.156.254:40671 requested a full tablet report, sending...
I20260807 18:21:15.197467 1356 ts_manager.cc:194] Registered new tserver with Master: 39dabdcfcc9640e08d04ecb15713e4fe (127.29.156.193:39845)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
I20260807 18:21:15.199889 1356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:55519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:15.231068 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:15.580049 1614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:15.580351 1614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:15.580504 1614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:15.588315 1614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:15.588429 1614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:15.588617 1614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:15.598537 1614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:15.599923 1614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:15.601305 1614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:15.615746 1622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:15.615948 1624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:15.615952 1627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:15.616782 1614 server_base.cc:1034] running on GCE node
I20260807 18:21:15.617475 1614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:15.618535 1614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:15.619765 1614 hybrid_clock.cc:648] HybridClock initialized: now 1786126875619709 us; error 54 us; skew 500 ppm
I20260807 18:21:15.620122 1614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:15 dist-test-slave-sb2z krb5kdc[1291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126875, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:15.623386 1614 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:15.625018 1614 webserver.cc:460] Webserver started at http://127.29.156.194:35649/ using document root <none> and password file <none>
I20260807 18:21:15.625761 1614 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:15.625910 1614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:15.626241 1614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:15.629388 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "b101366e7b25498395739d3a5530a837"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "22980cb69054f0b311b3a1451d72f0b4"
server_key_iv: "cf9db89ba7e3db22ef29e7cfac7ede1e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.630234 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "b101366e7b25498395739d3a5530a837"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "22980cb69054f0b311b3a1451d72f0b4"
server_key_iv: "cf9db89ba7e3db22ef29e7cfac7ede1e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.635831 1614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 18:21:15.639430 1636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.640828 1614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:15.641027 1614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "b101366e7b25498395739d3a5530a837"
format_stamp: "Formatted at 2026-08-07 18:21:15 on dist-test-slave-sb2z"
server_key: "22980cb69054f0b311b3a1451d72f0b4"
server_key_iv: "cf9db89ba7e3db22ef29e7cfac7ede1e"
server_key_version: "encryptionkey@0"
I20260807 18:21:15.641247 1614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:15.661772 1614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:15.666599 1614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:15.666939 1614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:15.668159 1614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:15.669785 1614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:15.669909 1614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.670037 1614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:15.670090 1614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:15.708046 1614 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43755
I20260807 18:21:15.708129 1788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43755 every 8 connection(s)
I20260807 18:21:15.709383 1614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:15.716521 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 1614
I20260807 18:21:15.716732 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:15.717312 30323 external_mini_cluster.cc:1428] Setting key 08b2269cba7eda993b998b6f3758da9e
Aug 07 18:21:15 dist-test-slave-sb2z krb5kdc[1291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126875, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:15.734112 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:15.712219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:51315 (local address 127.29.156.254:40671)
0807 18:21:15.712704 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:15.712709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:15.713634 (+ 925us) server_negotiation.cc:407] Connection header received
0807 18:21:15.715573 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:15.715579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:15.715669 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:15.715789 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:15.718047 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.719082 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.721696 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.722138 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.722248 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:15.726671 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:15.726680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:15.726688 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:15.726735 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:15.730175 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:15.731264 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:15.731268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:15.731270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:15.731405 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:15.732001 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:15.732005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:15.732007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:15.732399 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:15.732536 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:15.733578 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0807 18:21:15.733773 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":274,"threads_started":1}
I20260807 18:21:15.735939 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:15.712738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40671 (local address 127.29.156.194:51315)
0807 18:21:15.713463 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:15.713512 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:15.715246 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:15.716055 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:15.716102 (+ 47us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:15.716950 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:15.717836 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:15.717864 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.719265 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.719270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:15.721499 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:15.721519 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:15.722398 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:15.722408 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:15.722779 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:15.723578 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:15.723628 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:15.726413 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:15.730338 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:15.730351 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:15.730374 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:15.731135 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:15.731547 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:15.731555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:15.731562 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:15.731848 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:15.732558 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:15.732570 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:15.733097 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 18:21:15.733494 (+ 397us) client_negotiation.cc:240] Negotiation successful
0807 18:21:15.734207 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":270,"threads_started":1}
I20260807 18:21:15.737737 1790 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40671
I20260807 18:21:15.738117 1790 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:15.738973 1790 heartbeater.cc:511] Master 127.29.156.254:40671 requested a full tablet report, sending...
I20260807 18:21:15.740830 1356 ts_manager.cc:194] Registered new tserver with Master: b101366e7b25498395739d3a5530a837 (127.29.156.194:43755)
I20260807 18:21:15.741789 1356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:51315
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:15.781037 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:16.132220 1797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:16.132583 1797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:16.132740 1797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:16.140307 1797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:16.140465 1797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:16.140648 1797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:16.151147 1797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:16.152609 1797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:16.154034 1797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:16.168403 1805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:16.168668 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:16.168972 1804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:16.169483 1797 server_base.cc:1034] running on GCE node
I20260807 18:21:16.170297 1797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:16.171449 1797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:16.172730 1797 hybrid_clock.cc:648] HybridClock initialized: now 1786126876172664 us; error 58 us; skew 500 ppm
I20260807 18:21:16.173092 1797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:16 dist-test-slave-sb2z krb5kdc[1291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126876, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:16.176829 1797 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:16.178581 1797 webserver.cc:460] Webserver started at http://127.29.156.195:33997/ using document root <none> and password file <none>
I20260807 18:21:16.179335 1797 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:16.179508 1797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:16.179864 1797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:16.182974 1797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "5a659544c4ad418ab137d8a88e627d57"
format_stamp: "Formatted at 2026-08-07 18:21:16 on dist-test-slave-sb2z"
server_key: "cddbfcc012483b804f19edf14f26819f"
server_key_iv: "8d2cd912359ff95b3d4d77eea0c8ae27"
server_key_version: "encryptionkey@0"
I20260807 18:21:16.183904 1797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "5a659544c4ad418ab137d8a88e627d57"
format_stamp: "Formatted at 2026-08-07 18:21:16 on dist-test-slave-sb2z"
server_key: "cddbfcc012483b804f19edf14f26819f"
server_key_iv: "8d2cd912359ff95b3d4d77eea0c8ae27"
server_key_version: "encryptionkey@0"
I20260807 18:21:16.189548 1797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:21:16.193372 1815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:16.194926 1797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:21:16.195144 1797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "5a659544c4ad418ab137d8a88e627d57"
format_stamp: "Formatted at 2026-08-07 18:21:16 on dist-test-slave-sb2z"
server_key: "cddbfcc012483b804f19edf14f26819f"
server_key_iv: "8d2cd912359ff95b3d4d77eea0c8ae27"
server_key_version: "encryptionkey@0"
I20260807 18:21:16.195392 1797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:16.203365 1608 heartbeater.cc:503] Master 127.29.156.254:40671 was elected leader, sending a full tablet report...
I20260807 18:21:16.219199 1797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:16.224218 1797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:16.224474 1797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:16.225513 1797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:16.226922 1797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:16.227013 1797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:16.227173 1797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:16.227223 1797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:16.263396 1797 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:44135
I20260807 18:21:16.263455 1968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:44135 every 8 connection(s)
I20260807 18:21:16.264751 1797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:16.267627 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 1797
I20260807 18:21:16.267853 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:16.268425 30323 external_mini_cluster.cc:1428] Setting key e7f1d6ea386211aa6533c7db650cabb5
Aug 07 18:21:16 dist-test-slave-sb2z krb5kdc[1291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126876, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:16.288388 1973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:16.267845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33983 (local address 127.29.156.254:40671)
0807 18:21:16.268469 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:16.268477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:16.269723 (+ 1246us) server_negotiation.cc:407] Connection header received
0807 18:21:16.272171 (+ 2448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:16.272175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:16.272247 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:16.272355 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:16.274224 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:16.275015 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:16.277100 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:16.277525 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:16.277711 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:16.281639 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:16.281644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:16.281647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:16.281706 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:16.285134 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:16.285908 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:16.285912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:16.285915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:16.286053 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:16.286633 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:16.286636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:16.286638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:16.286911 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:16.287024 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:16.287876 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 18:21:16.288086 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":318,"threads_started":1}
I20260807 18:21:16.289422 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:16.268590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40671 (local address 127.29.156.195:33983)
0807 18:21:16.269537 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:16.269589 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:16.271857 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:16.272617 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:16.272626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:16.273264 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:16.273879 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:16.273902 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:16.275262 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:16.275267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:16.276929 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:16.276944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:16.277867 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:16.277874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:16.278143 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:16.278974 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:16.279000 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:16.281397 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:16.285307 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:16.285314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:16.285329 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:16.285783 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:16.286287 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:16.286291 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:16.286294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:16.286510 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:16.287020 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:16.287026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:16.287381 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 18:21:16.287695 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 18:21:16.288171 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":393,"threads_started":1}
I20260807 18:21:16.291059 1970 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40671
I20260807 18:21:16.291370 1970 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:16.292248 1970 heartbeater.cc:511] Master 127.29.156.254:40671 requested a full tablet report, sending...
I20260807 18:21:16.293740 1356 ts_manager.cc:194] Registered new tserver with Master: 5a659544c4ad418ab137d8a88e627d57 (127.29.156.195:44135)
I20260807 18:21:16.294696 1356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:33983
I20260807 18:21:16.297358 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:21:16.338841 1973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:16.336441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.29.156.254:40671)
0807 18:21:16.336621 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:16.336626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:16.337053 (+ 427us) server_negotiation.cc:407] Connection header received
0807 18:21:16.337133 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:16.337137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:16.337216 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:16.337316 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:16.338620 (+ 1304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54072: BlockingRecv error: recv got EOF from 127.0.0.1:54072 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:21:16.341914 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1437
W20260807 18:21:16.345028 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1437: Not authorized: Client connection negotiation failed: client connection to 127.29.156.193:39845: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:21:16.345198 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 1437
I20260807 18:21:16.371037 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1614
W20260807 18:21:16.373997 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1614: Not authorized: Client connection negotiation failed: client connection to 127.29.156.194:43755: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:21:16.374166 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 1614
I20260807 18:21:16.399571 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1797
W20260807 18:21:16.402645 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1797: Not authorized: Client connection negotiation failed: client connection to 127.29.156.195:44135: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:21:16.402796 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 1797
I20260807 18:21:16.425347 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 1313
I20260807 18:21:16.427073 1392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:16.534936 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 1313
2026-08-07T18:21:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4128 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:16 dist-test-slave-sb2z krb5kdc[1994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:16 dist-test-slave-sb2z krb5kdc[1994](info): set up 2 sockets
Aug 07 18:21:16 dist-test-slave-sb2z krb5kdc[1994](info): commencing operation
krb5kdc: starting...
W20260807 18:21:17.891381 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[1994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:18.143818 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46375
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:18.486630 2011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:18.486936 2011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:18.487030 2011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:18.494700 2011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:18.494843 2011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:18.494891 2011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:18.494999 2011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:18.495143 2011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:18.505749 2011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:18.507102 2011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:18.508481 2011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:18.520995 2019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:18.520980 2021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:18.521018 2023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:18.522137 2011 server_base.cc:1034] running on GCE node
I20260807 18:21:18.522905 2011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:18.524384 2011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:18.525617 2011 hybrid_clock.cc:648] HybridClock initialized: now 1786126878525578 us; error 51 us; skew 500 ppm
I20260807 18:21:18.525982 2011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[1994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126878, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:18.532485 2011 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:18.534073 2011 webserver.cc:460] Webserver started at http://127.29.156.254:34765/ using document root <none> and password file <none>
I20260807 18:21:18.534722 2011 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:18.534829 2011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:18.535115 2011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:18.538266 2011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "819debd7a22e43178051eeed366efc99"
format_stamp: "Formatted at 2026-08-07 18:21:18 on dist-test-slave-sb2z"
server_key: "7c3ba23a369a8af1644f887403026df4"
server_key_iv: "e193cde3dd78a5a46ce9bd3b0642a5fb"
server_key_version: "encryptionkey@0"
I20260807 18:21:18.539055 2011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "819debd7a22e43178051eeed366efc99"
format_stamp: "Formatted at 2026-08-07 18:21:18 on dist-test-slave-sb2z"
server_key: "7c3ba23a369a8af1644f887403026df4"
server_key_iv: "e193cde3dd78a5a46ce9bd3b0642a5fb"
server_key_version: "encryptionkey@0"
I20260807 18:21:18.544394 2011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:21:18.547896 2032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:18.549345 2011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:18.549496 2011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "819debd7a22e43178051eeed366efc99"
format_stamp: "Formatted at 2026-08-07 18:21:18 on dist-test-slave-sb2z"
server_key: "7c3ba23a369a8af1644f887403026df4"
server_key_iv: "e193cde3dd78a5a46ce9bd3b0642a5fb"
server_key_version: "encryptionkey@0"
I20260807 18:21:18.549676 2011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:18.584466 2011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:18.589748 2011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:18.590018 2011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:18.610661 2011 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:46375
I20260807 18:21:18.610713 2107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:46375 every 8 connection(s)
I20260807 18:21:18.612313 2011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:18.616276 2109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:18.617717 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2011
I20260807 18:21:18.617985 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:18.618897 30323 external_mini_cluster.cc:1428] Setting key 561188101cb0a0db4e65a25e292847de
I20260807 18:21:18.632063 2109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99: Bootstrap starting.
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[1994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:18.636034 2109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:18.637329 2109 log.cc:824] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:18.640533 2109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99: No bootstrap required, opened a new log
I20260807 18:21:18.644258 2112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:18.621097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.29.156.254:46375)
0807 18:21:18.622078 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:18.622091 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:18.622140 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:21:18.624006 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:18.624044 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:18.624504 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:18.624872 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:18.626782 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:18.628387 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:18.630258 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:18.630958 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:18.631161 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:18.636011 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:18.636023 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:18.636038 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:18.636076 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:18.639758 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:18.640464 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:18.640478 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:18.640484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:18.640749 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:18.641200 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:18.641205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:18.641208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:18.641794 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:18.642134 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:18.642644 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 18:21:18.643116 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":724,"thread_start_us":482,"threads_started":1}
I20260807 18:21:18.648413 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:21:18.649564 2109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } }
I20260807 18:21:18.649950 2109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:18.650066 2109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819debd7a22e43178051eeed366efc99, State: Initialized, Role: FOLLOWER
I20260807 18:21:18.650616 2109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } }
I20260807 18:21:18.650854 2109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:18.651000 2109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:18.651165 2109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:18.653496 2109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } }
I20260807 18:21:18.653999 2109 leader_election.cc:302] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 819debd7a22e43178051eeed366efc99; no voters:
I20260807 18:21:18.654736 2109 leader_election.cc:288] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:18.654866 2114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:18.657444 2109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:18.658567 2114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [term 1 LEADER]: Becoming Leader. State: Replica: 819debd7a22e43178051eeed366efc99, State: Running, Role: LEADER
I20260807 18:21:18.659229 2114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } }
I20260807 18:21:18.663060 2116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 819debd7a22e43178051eeed366efc99. Latest consensus state: current_term: 1 leader_uuid: "819debd7a22e43178051eeed366efc99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } } }
I20260807 18:21:18.663458 2116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:18.664153 2115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "819debd7a22e43178051eeed366efc99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819debd7a22e43178051eeed366efc99" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46375 } } }
I20260807 18:21:18.664474 2115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:18.675458 2128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:18.686671 2128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:18.695961 2128 catalog_manager.cc:1347] Generated new cluster ID: d214ef78e086453c85494faa10c10dc3
I20260807 18:21:18.696111 2128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:18.719781 2128 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:18.721624 2128 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:18.727129 2112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:18.717520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.29.156.254:46375)
0807 18:21:18.717713 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:18.717720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:18.718196 (+ 476us) server_negotiation.cc:407] Connection header received
0807 18:21:18.718473 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:18.718480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:18.718571 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:18.718700 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:18.720116 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:18.721251 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:18.723274 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:18.723937 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:18.724059 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:18.725363 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:18.725369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:18.725374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:18.726210 (+ 836us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:18.726290 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:18.726319 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:18.726424 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:18.726605 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 18:21:18.726740 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260807 18:21:18.733073 2128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 819debd7a22e43178051eeed366efc99: Generated new TSK 0
I20260807 18:21:18.734061 2128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:21:18.742765 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2011
I20260807 18:21:18.744344 2102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:18.837040 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2011
2026-08-07T18:21:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2302 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[2150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[2150](info): set up 2 sockets
Aug 07 18:21:18 dist-test-slave-sb2z krb5kdc[2150](info): commencing operation
krb5kdc: starting...
W20260807 18:21:20.183284 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:20 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:20.439831 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43955
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:20.793438 2166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:20.793778 2166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:20.794097 2166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:20.801841 2166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:20.801987 2166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:20.802039 2166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:20.802186 2166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:20.802258 2166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:20.813678 2166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:20.815198 2166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:20.816606 2166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:20.829875 2179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:20.829985 2177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:20.830020 2176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:20.831964 2166 server_base.cc:1034] running on GCE node
I20260807 18:21:20.832882 2166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:20.834475 2166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:20.835879 2166 hybrid_clock.cc:648] HybridClock initialized: now 1786126880835813 us; error 62 us; skew 500 ppm
I20260807 18:21:20.836275 2166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:20 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126880, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:20.840096 2166 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:20.841794 2166 webserver.cc:460] Webserver started at http://127.29.156.254:33909/ using document root <none> and password file <none>
I20260807 18:21:20.842490 2166 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:20.842609 2166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:20.842871 2166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:20.846103 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "03bdf019ded9432db105c582f09c564e"
format_stamp: "Formatted at 2026-08-07 18:21:20 on dist-test-slave-sb2z"
server_key: "3ff1848343e5a5e9e4aff0ce79756798"
server_key_iv: "adcc5877b3400d6629c826568d4186fa"
server_key_version: "encryptionkey@0"
I20260807 18:21:20.846925 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "03bdf019ded9432db105c582f09c564e"
format_stamp: "Formatted at 2026-08-07 18:21:20 on dist-test-slave-sb2z"
server_key: "3ff1848343e5a5e9e4aff0ce79756798"
server_key_iv: "adcc5877b3400d6629c826568d4186fa"
server_key_version: "encryptionkey@0"
I20260807 18:21:20.852880 2166 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:21:20.857040 2189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:20.858675 2166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:20.858914 2166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "03bdf019ded9432db105c582f09c564e"
format_stamp: "Formatted at 2026-08-07 18:21:20 on dist-test-slave-sb2z"
server_key: "3ff1848343e5a5e9e4aff0ce79756798"
server_key_iv: "adcc5877b3400d6629c826568d4186fa"
server_key_version: "encryptionkey@0"
I20260807 18:21:20.859215 2166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:20.879519 2166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:20.884302 2166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:20.884641 2166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:20.905489 2166 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43955
I20260807 18:21:20.905560 2253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43955 every 8 connection(s)
I20260807 18:21:20.907083 2166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:20.911358 2254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:20.916779 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2166
I20260807 18:21:20.917042 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:20.917966 30323 external_mini_cluster.cc:1428] Setting key 15dbaea969cf8fc3ce85dae4535f4db2
I20260807 18:21:20.925390 2254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e: Bootstrap starting.
I20260807 18:21:20.929507 2254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:20.930824 2254 log.cc:824] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:21:20 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:20.934669 2254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e: No bootstrap required, opened a new log
I20260807 18:21:20.944834 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:20.920528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.29.156.254:43955)
0807 18:21:20.921490 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:20.921503 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:20.921553 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:21:20.923405 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:20.923430 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:20.923904 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:20.924275 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:20.926285 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:20.927728 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:20.929731 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:20.930508 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:20.930658 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:20.935665 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:20.935674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:20.935689 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:20.935725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:20.939370 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:20.940277 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:20.940293 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:20.940300 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:20.940505 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:20.941339 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:20.941348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:20.941351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:20.942082 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:20.942503 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:20.943075 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 18:21:20.943681 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":434,"threads_started":1}
I20260807 18:21:20.944744 2254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } }
I20260807 18:21:20.945300 2254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:20.945461 2254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03bdf019ded9432db105c582f09c564e, State: Initialized, Role: FOLLOWER
I20260807 18:21:20.946199 2254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } }
I20260807 18:21:20.946481 2254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:20.946672 2254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:20.946935 2254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:20.949944 2254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } }
I20260807 18:21:20.950527 2254 leader_election.cc:302] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03bdf019ded9432db105c582f09c564e; no voters:
I20260807 18:21:20.951301 2254 leader_election.cc:288] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:20.951433 2261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:20.952365 2261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [term 1 LEADER]: Becoming Leader. State: Replica: 03bdf019ded9432db105c582f09c564e, State: Running, Role: LEADER
I20260807 18:21:20.952993 2261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } }
I20260807 18:21:20.954023 2254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:20.956298 2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03bdf019ded9432db105c582f09c564e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } } }
I20260807 18:21:20.956713 2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:20.956434 2265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03bdf019ded9432db105c582f09c564e. Latest consensus state: current_term: 1 leader_uuid: "03bdf019ded9432db105c582f09c564e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bdf019ded9432db105c582f09c564e" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43955 } } }
I20260807 18:21:20.956939 2265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:20.958850 2273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:20.966867 2273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:20.977495 2273 catalog_manager.cc:1347] Generated new cluster ID: 4d4f1f1ef7ec41589d3c63e3ec68ed6f
I20260807 18:21:20.977685 2273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:20.998481 2273 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:21.000320 2273 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:21.008970 2273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03bdf019ded9432db105c582f09c564e: Generated new TSK 0
I20260807 18:21:21.009898 2273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:21.083727 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:21.464717 2291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:21.464998 2291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:21.465200 2291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:21.472664 2291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:21.472781 2291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:21.472986 2291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:21.483161 2291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:21.484673 2291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:21.486097 2291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:21.500782 2296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:21.500734 2297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:21.500692 2301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:21.505276 2291 server_base.cc:1034] running on GCE node
I20260807 18:21:21.506039 2291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:21.507102 2291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:21.508333 2291 hybrid_clock.cc:648] HybridClock initialized: now 1786126881508273 us; error 48 us; skew 500 ppm
I20260807 18:21:21.508678 2291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:21 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126881, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:21.512657 2291 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:21.514319 2291 webserver.cc:460] Webserver started at http://127.29.156.193:44147/ using document root <none> and password file <none>
I20260807 18:21:21.515147 2291 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:21.515319 2291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:21.515738 2291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:21.519065 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "e8e5b41b73a244db8c664812659d3722"
format_stamp: "Formatted at 2026-08-07 18:21:21 on dist-test-slave-sb2z"
server_key: "714237775c913d2b8b80512c3afb6eda"
server_key_iv: "358cb9bfd22bd9438bb4a7f055f4f6a4"
server_key_version: "encryptionkey@0"
I20260807 18:21:21.520021 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "e8e5b41b73a244db8c664812659d3722"
format_stamp: "Formatted at 2026-08-07 18:21:21 on dist-test-slave-sb2z"
server_key: "714237775c913d2b8b80512c3afb6eda"
server_key_iv: "358cb9bfd22bd9438bb4a7f055f4f6a4"
server_key_version: "encryptionkey@0"
I20260807 18:21:21.526041 2291 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:21:21.530131 2308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:21.531863 2291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:21:21.532074 2291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "e8e5b41b73a244db8c664812659d3722"
format_stamp: "Formatted at 2026-08-07 18:21:21 on dist-test-slave-sb2z"
server_key: "714237775c913d2b8b80512c3afb6eda"
server_key_iv: "358cb9bfd22bd9438bb4a7f055f4f6a4"
server_key_version: "encryptionkey@0"
I20260807 18:21:21.532301 2291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:21.556690 2291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:21.561780 2291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:21.562248 2291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:21.563405 2291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:21.564879 2291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:21.564989 2291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:21.565155 2291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:21.565231 2291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:21.603695 2291 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:40691
I20260807 18:21:21.603786 2469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:40691 every 8 connection(s)
I20260807 18:21:21.605082 2291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:21.613677 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2291
I20260807 18:21:21.614002 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:21.614697 30323 external_mini_cluster.cc:1428] Setting key 5b681d5d76bb1701a1aa7b0610d144f0
Aug 07 18:21:21 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126881, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:21.628018 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:21.608071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:49575 (local address 127.29.156.254:43955)
0807 18:21:21.608570 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:21.608575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:21.609490 (+ 915us) server_negotiation.cc:407] Connection header received
0807 18:21:21.611488 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:21.611494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:21.611630 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:21.611760 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:21.613345 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:21.614222 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:21.616189 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:21.616752 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:21.616903 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:21.620969 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:21.620975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:21.620979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:21.621017 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:21.623870 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:21.624917 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:21.624923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:21.624927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:21.625113 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:21.625695 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:21.625702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:21.625708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:21.626182 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:21.626350 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:21.627203 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 18:21:21.627480 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260807 18:21:21.629021 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:21.608525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43955 (local address 127.29.156.193:49575)
0807 18:21:21.609301 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:21.609353 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:21.611191 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:21.612018 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:21.612027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:21.612570 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:21.613146 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:21.613164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:21.614387 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:21.614391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:21.616005 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:21.616022 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:21.617017 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:21.617022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:21.617223 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:21.617986 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:21.618014 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:21.620679 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:21.624032 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:21.624044 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:21.624066 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:21.624764 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:21.625242 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:21.625250 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:21.625255 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:21.625541 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:21.626358 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:21.626365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:21.626777 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 18:21:21.627074 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 18:21:21.627648 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":319,"threads_started":1}
I20260807 18:21:21.630864 2471 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43955
I20260807 18:21:21.631227 2471 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:21.632290 2471 heartbeater.cc:511] Master 127.29.156.254:43955 requested a full tablet report, sending...
I20260807 18:21:21.634807 2208 ts_manager.cc:194] Registered new tserver with Master: e8e5b41b73a244db8c664812659d3722 (127.29.156.193:40691)
I20260807 18:21:21.636669 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:49575
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:21.675302 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:22.038460 2480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:22.038821 2480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:22.038990 2480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:22.046641 2480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:22.046803 2480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:22.046996 2480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:22.057435 2480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:22.058835 2480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:22.060405 2480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:22.074292 2489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:22.074328 2488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:22.074371 2491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:22.075166 2480 server_base.cc:1034] running on GCE node
I20260807 18:21:22.075875 2480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:22.076874 2480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:22.078153 2480 hybrid_clock.cc:648] HybridClock initialized: now 1786126882078123 us; error 72 us; skew 500 ppm
I20260807 18:21:22.078514 2480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:22.082243 2480 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:22.083930 2480 webserver.cc:460] Webserver started at http://127.29.156.194:32931/ using document root <none> and password file <none>
I20260807 18:21:22.084646 2480 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:22.084800 2480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:22.085125 2480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:22.088182 2480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "2b72db90cbbb46f89f77f5c1013c5582"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "7ce2fcaf483c3f7c80c007e22d2cb296"
server_key_iv: "6fe9d4c693c337ae0335f3b1a4959575"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.088984 2480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "2b72db90cbbb46f89f77f5c1013c5582"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "7ce2fcaf483c3f7c80c007e22d2cb296"
server_key_iv: "6fe9d4c693c337ae0335f3b1a4959575"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.094520 2480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:21:22.098601 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.100188 2480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:22.100374 2480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "2b72db90cbbb46f89f77f5c1013c5582"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "7ce2fcaf483c3f7c80c007e22d2cb296"
server_key_iv: "6fe9d4c693c337ae0335f3b1a4959575"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.100591 2480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:22.117792 2480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:22.122293 2480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:22.122629 2480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:22.123777 2480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:22.125190 2480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:22.125301 2480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.125463 2480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:22.125540 2480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.164392 2480 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43289
I20260807 18:21:22.164397 2655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43289 every 8 connection(s)
I20260807 18:21:22.165791 2480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:22.173074 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2480
I20260807 18:21:22.173305 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:22.173954 30323 external_mini_cluster.cc:1428] Setting key 56c8d68562161556aaea2dc8070698bc
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:22.191135 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.168794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:48379 (local address 127.29.156.254:43955)
0807 18:21:22.169299 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.169306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.170387 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 18:21:22.172526 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.172531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.172603 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.172708 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.175046 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.176043 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.177817 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.178685 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.178847 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.183699 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.183709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.183716 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.183766 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.187029 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.187985 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.187991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.187995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.188190 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.188688 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.188697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.188704 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.189161 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.189340 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.190233 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.190542 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260807 18:21:22.191831 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.169373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43955 (local address 127.29.156.194:48379)
0807 18:21:22.170202 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:22.170245 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:22.172225 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:22.173070 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:22.173083 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:22.173964 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:22.174833 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:22.174858 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.176226 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.176232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:22.177632 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:22.177646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.178949 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.178954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:22.179192 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.180162 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:22.180204 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:22.183383 (+ 3179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:22.187206 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:22.187218 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:22.187237 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:22.187831 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:22.188324 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:22.188328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:22.188330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:22.188551 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:22.189312 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:22.189319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:22.189817 (+ 498us) client_negotiation.cc:769] Sending connection context
0807 18:21:22.190072 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 18:21:22.190494 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":307,"threads_started":1}
I20260807 18:21:22.193722 2656 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43955
I20260807 18:21:22.194072 2656 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:22.194927 2656 heartbeater.cc:511] Master 127.29.156.254:43955 requested a full tablet report, sending...
I20260807 18:21:22.196594 2208 ts_manager.cc:194] Registered new tserver with Master: 2b72db90cbbb46f89f77f5c1013c5582 (127.29.156.194:43289)
I20260807 18:21:22.197578 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:48379
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:22.235411 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:22.580662 2665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:22.581032 2665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:22.581234 2665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:22.588712 2665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:22.588877 2665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:22.589073 2665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:22.598985 2665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:22.600410 2665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:22.601869 2665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:22.615893 2673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:22.615880 2671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:22.616146 2670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:22.617381 2665 server_base.cc:1034] running on GCE node
I20260807 18:21:22.618014 2665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:22.619028 2665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:22.620282 2665 hybrid_clock.cc:648] HybridClock initialized: now 1786126882620241 us; error 41 us; skew 500 ppm
I20260807 18:21:22.620574 2665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:22.623962 2665 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:22.625532 2665 webserver.cc:460] Webserver started at http://127.29.156.195:43003/ using document root <none> and password file <none>
I20260807 18:21:22.626194 2665 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:22.626303 2665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:22.626569 2665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:22.630128 2665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "fb878cf7bf79490db73242691219fb2a"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "3faf704d453431872f20dd0a4d75304e"
server_key_iv: "6403fbf249d8b429e6a4adf277784022"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.630983 2665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "fb878cf7bf79490db73242691219fb2a"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "3faf704d453431872f20dd0a4d75304e"
server_key_iv: "6403fbf249d8b429e6a4adf277784022"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.636466 2665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:21:22.640476 2682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.640762 2471 heartbeater.cc:503] Master 127.29.156.254:43955 was elected leader, sending a full tablet report...
I20260807 18:21:22.642211 2665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:21:22.642402 2665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "fb878cf7bf79490db73242691219fb2a"
format_stamp: "Formatted at 2026-08-07 18:21:22 on dist-test-slave-sb2z"
server_key: "3faf704d453431872f20dd0a4d75304e"
server_key_iv: "6403fbf249d8b429e6a4adf277784022"
server_key_version: "encryptionkey@0"
I20260807 18:21:22.642642 2665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:22.679394 2665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:22.684177 2665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:22.684449 2665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:22.685431 2665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:22.686801 2665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:22.686879 2665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.686976 2665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:22.687031 2665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:22.724293 2665 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:46815
I20260807 18:21:22.724339 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:46815 every 8 connection(s)
I20260807 18:21:22.725732 2665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:22.732313 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2665
I20260807 18:21:22.732570 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:22.733167 30323 external_mini_cluster.cc:1428] Setting key 15855a676f1e1bad050af720675f1a64
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:22.749617 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.728531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:59303 (local address 127.29.156.254:43955)
0807 18:21:22.729050 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.729055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.729999 (+ 944us) server_negotiation.cc:407] Connection header received
0807 18:21:22.732167 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.732171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.732263 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.732368 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.734450 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.735512 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.737858 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.738335 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.738447 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.742400 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.742405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.742409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.742452 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.745458 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.746454 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.746460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.746462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.746630 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.747253 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.747257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.747260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.747759 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.747938 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.748911 (+ 973us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.749167 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":259,"threads_started":1}
I20260807 18:21:22.750797 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.728998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43955 (local address 127.29.156.195:59303)
0807 18:21:22.729800 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:22.729849 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:22.731759 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:22.732712 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:22.732724 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:22.733558 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:22.734250 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:22.734269 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.735755 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.735760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:22.737697 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:22.737711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.738586 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.738591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:22.738796 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.739621 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:22.739654 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:22.742131 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:22.745648 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:22.745656 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:22.745672 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:22.746264 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:22.746790 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:22.746800 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:22.746803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:22.747085 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:22.747924 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:22.747942 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:22.748385 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 18:21:22.748773 (+ 388us) client_negotiation.cc:240] Negotiation successful
0807 18:21:22.749361 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":324,"threads_started":1}
I20260807 18:21:22.752409 2837 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43955
I20260807 18:21:22.752804 2837 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:22.753754 2837 heartbeater.cc:511] Master 127.29.156.254:43955 requested a full tablet report, sending...
I20260807 18:21:22.755743 2208 ts_manager.cc:194] Registered new tserver with Master: fb878cf7bf79490db73242691219fb2a (127.29.156.195:46815)
I20260807 18:21:22.756632 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:59303
I20260807 18:21:22.763155 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:22.826812 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.810709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.29.156.254:43955)
0807 18:21:22.810987 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.810996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.811486 (+ 490us) server_negotiation.cc:407] Connection header received
0807 18:21:22.811754 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.811760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.811850 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.811954 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.813452 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.814323 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.816083 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.816560 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.816670 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.821081 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.821087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.821090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.821130 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.823997 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.824703 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.824707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.824709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.824834 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.825255 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.825259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.825261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.825619 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.825903 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.826241 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.826428 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 18:21:22.827612 2208 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36622
I20260807 18:21:22.830011 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2291
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:22.848073 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.831416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.29.156.193:40691)
0807 18:21:22.832040 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.832048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.832073 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:21:22.832134 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.832141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.832363 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.832513 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.833858 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.834976 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.836589 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.837265 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.837377 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.841582 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.841595 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.841610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.841646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.844795 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.845568 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.845576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.845578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.845720 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.846304 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.846309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.846311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.846758 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.846999 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.847466 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.847764 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":309,"threads_started":1}
W20260807 18:21:22.849371 2462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41522
W20260807 18:21:22.850526 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2291: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:22.850790 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2291
I20260807 18:21:22.876343 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2480
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:22.894034 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.877323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.29.156.194:43289)
0807 18:21:22.877946 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.877954 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.877979 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:21:22.878049 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.878055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.878261 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.878491 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.879828 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.880872 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.882298 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.882977 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.883097 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.887733 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.887750 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.887766 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.887810 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.890828 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.891826 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.891836 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.891839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.892003 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.892374 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.892378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.892380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.892783 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.893083 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.893348 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.893658 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":302,"threads_started":1}
W20260807 18:21:22.895439 2648 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42832
W20260807 18:21:22.896785 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2480: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:22.896939 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2480
I20260807 18:21:22.924216 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2665
Aug 07 18:21:22 dist-test-slave-sb2z krb5kdc[2150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126882, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:22.942052 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:22.925261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.29.156.195:46815)
0807 18:21:22.926058 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:22.926065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:22.926096 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:21:22.926157 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:22.926163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:22.926371 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:22.926553 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:22.928055 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.929137 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.930387 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:22.930986 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:22.931131 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:22.935973 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:22.935987 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:22.936003 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:22.936045 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:22.938976 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.939774 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.939782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.939784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.939916 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:22.940323 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:22.940327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:22.940329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:22.940696 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:22.941050 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:22.941359 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 18:21:22.941660 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":420,"threads_started":1}
W20260807 18:21:22.943605 2830 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59806
W20260807 18:21:22.944886 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:21:22.945135 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2665
I20260807 18:21:22.970711 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2166
I20260807 18:21:22.972093 2244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:23.086084 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2166
2026-08-07T18:21:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4255 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:23 dist-test-slave-sb2z krb5kdc[2862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:23 dist-test-slave-sb2z krb5kdc[2862](info): set up 2 sockets
Aug 07 18:21:23 dist-test-slave-sb2z krb5kdc[2862](info): commencing operation
krb5kdc: starting...
W20260807 18:21:24.422926 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:24 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:24.688552 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37677
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:25.034761 2881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:25.035198 2881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:25.035384 2881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:25.043084 2881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:25.043251 2881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:25.043342 2881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:25.043454 2881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:25.043545 2881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:25.053916 2881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:25.055373 2881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:25.056906 2881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:25.069581 2887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:25.069727 2891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:25.069770 2889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:25.070705 2881 server_base.cc:1034] running on GCE node
I20260807 18:21:25.071585 2881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:25.073215 2881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:25.074501 2881 hybrid_clock.cc:648] HybridClock initialized: now 1786126885074411 us; error 84 us; skew 500 ppm
I20260807 18:21:25.074910 2881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:25 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126885, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:25.083132 2881 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:25.085022 2881 webserver.cc:460] Webserver started at http://127.29.156.254:38947/ using document root <none> and password file <none>
I20260807 18:21:25.085793 2881 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:25.085959 2881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:25.086311 2881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:25.089501 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "fc15382c0af04b72a42f8981f7316ffb"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "5c707c3a4bfe8047b58cfadca70c7583"
server_key_iv: "f95f033c3566c0dee0829518d1363a2f"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.090391 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "fc15382c0af04b72a42f8981f7316ffb"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "5c707c3a4bfe8047b58cfadca70c7583"
server_key_iv: "f95f033c3566c0dee0829518d1363a2f"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.096069 2881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:21:25.100152 2900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:25.101768 2881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:25.101994 2881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "fc15382c0af04b72a42f8981f7316ffb"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "5c707c3a4bfe8047b58cfadca70c7583"
server_key_iv: "f95f033c3566c0dee0829518d1363a2f"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.102317 2881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:25.125097 2881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:25.129895 2881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:25.130227 2881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:25.150084 2881 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37677
I20260807 18:21:25.150182 2973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37677 every 8 connection(s)
I20260807 18:21:25.151495 2881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:25.155113 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 2881
I20260807 18:21:25.155375 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:25.155623 2974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:25.156150 30323 external_mini_cluster.cc:1428] Setting key 765a561061d4aa6d9fa6d0f68d265fa9
I20260807 18:21:25.170809 2974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb: Bootstrap starting.
Aug 07 18:21:25 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:25.175040 2974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:25.176486 2974 log.cc:824] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:25.180328 2974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb: No bootstrap required, opened a new log
I20260807 18:21:25.181667 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:25.158168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.29.156.254:37677)
0807 18:21:25.159038 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:25.159051 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:25.159095 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:21:25.160717 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:25.160740 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:25.161138 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:25.161487 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:25.163481 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.164846 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.166592 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.167313 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.167446 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:25.172353 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:25.172370 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:25.172392 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:25.172472 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:25.176631 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:25.177682 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:25.177689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:25.177695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:25.177910 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:25.178350 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:25.178355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:25.178358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:25.179047 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:25.179342 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:25.179829 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 18:21:25.180261 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":414,"threads_started":1}
I20260807 18:21:25.188849 2974 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } }
I20260807 18:21:25.189213 2974 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:25.189342 2974 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc15382c0af04b72a42f8981f7316ffb, State: Initialized, Role: FOLLOWER
I20260807 18:21:25.189834 2974 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } }
I20260807 18:21:25.190057 2974 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:25.190156 2974 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:25.190281 2974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:25.192883 2974 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } }
I20260807 18:21:25.193349 2974 leader_election.cc:302] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc15382c0af04b72a42f8981f7316ffb; no voters:
I20260807 18:21:25.193926 2974 leader_election.cc:288] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:25.194061 2980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:25.194851 2980 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [term 1 LEADER]: Becoming Leader. State: Replica: fc15382c0af04b72a42f8981f7316ffb, State: Running, Role: LEADER
I20260807 18:21:25.195463 2980 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } }
I20260807 18:21:25.196240 2974 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:25.199187 2981 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc15382c0af04b72a42f8981f7316ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } } }
I20260807 18:21:25.199604 2981 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:25.200356 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc15382c0af04b72a42f8981f7316ffb. Latest consensus state: current_term: 1 leader_uuid: "fc15382c0af04b72a42f8981f7316ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc15382c0af04b72a42f8981f7316ffb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37677 } } }
I20260807 18:21:25.200742 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:25.202754 2991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:25.207844 2991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:25.216813 2991 catalog_manager.cc:1347] Generated new cluster ID: b63ee7eb1bc749e386c41783a802d39e
I20260807 18:21:25.216948 2991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:25.247359 2991 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:25.248811 2991 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:25.268314 2991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc15382c0af04b72a42f8981f7316ffb: Generated new TSK 0
I20260807 18:21:25.269271 2991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:25.335505 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:25.687302 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:25.687767 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:25.687955 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:25.695380 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:25.695494 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:25.695674 3007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:25.706435 3007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:25.707862 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:25.709357 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:25.723150 3013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:25.723220 3014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:25.723272 3017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:25.724189 3007 server_base.cc:1034] running on GCE node
I20260807 18:21:25.724865 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:25.726029 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:25.727295 3007 hybrid_clock.cc:648] HybridClock initialized: now 1786126885727267 us; error 78 us; skew 500 ppm
I20260807 18:21:25.727795 3007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:25 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126885, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:25.731703 3007 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:25.733693 3007 webserver.cc:460] Webserver started at http://127.29.156.193:34883/ using document root <none> and password file <none>
I20260807 18:21:25.734499 3007 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:25.734668 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:25.735057 3007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:25.738473 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "46ef3f7625e84a27be129ca328b23be2"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "d8fc33f690dc1536ac7ba3447fb385f2"
server_key_iv: "568a3549c9db6f380d27cfdf6b3f6c58"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.739394 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "46ef3f7625e84a27be129ca328b23be2"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "d8fc33f690dc1536ac7ba3447fb385f2"
server_key_iv: "568a3549c9db6f380d27cfdf6b3f6c58"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.745299 3007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:21:25.749351 3026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:25.751156 3007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 18:21:25.751370 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "46ef3f7625e84a27be129ca328b23be2"
format_stamp: "Formatted at 2026-08-07 18:21:25 on dist-test-slave-sb2z"
server_key: "d8fc33f690dc1536ac7ba3447fb385f2"
server_key_iv: "568a3549c9db6f380d27cfdf6b3f6c58"
server_key_version: "encryptionkey@0"
I20260807 18:21:25.751725 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:25.794701 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:25.799682 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:25.800025 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:25.801352 3007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:25.802918 3007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:25.803036 3007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:25.803196 3007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:25.803274 3007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:25.840391 3007 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:44475
I20260807 18:21:25.840512 3173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:44475 every 8 connection(s)
I20260807 18:21:25.841812 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:25.852203 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3007
I20260807 18:21:25.852479 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:25.853344 30323 external_mini_cluster.cc:1428] Setting key f2d619dcbaf63f1c8651896e5599afd8
Aug 07 18:21:25 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126885, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:25.865509 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:25.844763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40973 (local address 127.29.156.254:37677)
0807 18:21:25.845317 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:25.845323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:25.846380 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 18:21:25.848572 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:25.848585 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:25.848665 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:25.848769 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:25.850409 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.851191 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.853897 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.854529 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.854699 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:25.859090 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:25.859095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:25.859099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:25.859134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:25.861881 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:25.862722 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:25.862727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:25.862729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:25.862865 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:25.863345 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:25.863349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:25.863351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:25.863738 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:25.863904 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:25.864842 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 18:21:25.865139 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
I20260807 18:21:25.866730 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:25.845395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37677 (local address 127.29.156.193:40973)
0807 18:21:25.846197 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:25.846247 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:25.848243 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:25.849037 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:25.849046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:25.849637 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:25.850221 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:25.850239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.851409 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.851414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:25.853715 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:25.853733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:25.854792 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:25.854798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:25.855014 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:25.855860 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:25.855891 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:25.858822 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:25.862044 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:25.862055 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:25.862074 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:25.862575 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:25.862985 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:25.862990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:25.862994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:25.863211 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:25.863881 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:25.863888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:25.864276 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:21:25.864633 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 18:21:25.865104 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":299,"threads_started":1}
I20260807 18:21:25.868500 3174 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37677
I20260807 18:21:25.868854 3174 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:25.869648 3174 heartbeater.cc:511] Master 127.29.156.254:37677 requested a full tablet report, sending...
I20260807 18:21:25.872092 2925 ts_manager.cc:194] Registered new tserver with Master: 46ef3f7625e84a27be129ca328b23be2 (127.29.156.193:44475)
I20260807 18:21:25.873591 2925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:40973
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:25.915763 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:26.261943 3182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:26.262295 3182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:26.262457 3182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:26.269814 3182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:26.269975 3182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:26.270155 3182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:26.280367 3182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:26.281880 3182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:26.283277 3182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:26.297638 3193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:26.297664 3189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:26.298040 3188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:26.298435 3182 server_base.cc:1034] running on GCE node
I20260807 18:21:26.299085 3182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:26.300209 3182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:26.301484 3182 hybrid_clock.cc:648] HybridClock initialized: now 1786126886301368 us; error 59 us; skew 500 ppm
I20260807 18:21:26.301857 3182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:26 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:26.305382 3182 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:26.307085 3182 webserver.cc:460] Webserver started at http://127.29.156.194:38645/ using document root <none> and password file <none>
I20260807 18:21:26.307873 3182 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:26.307983 3182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:26.308235 3182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:26.311204 3182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "11d2be0faf4e44819f0f2bad55224749"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "1e8d49e7794137401dfd12b6bd2f8689"
server_key_iv: "00a0f898dca58e0a0c9c46d5fa99a047"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.312052 3182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "11d2be0faf4e44819f0f2bad55224749"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "1e8d49e7794137401dfd12b6bd2f8689"
server_key_iv: "00a0f898dca58e0a0c9c46d5fa99a047"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.317492 3182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:21:26.321156 3203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.322506 3182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:26.322644 3182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "11d2be0faf4e44819f0f2bad55224749"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "1e8d49e7794137401dfd12b6bd2f8689"
server_key_iv: "00a0f898dca58e0a0c9c46d5fa99a047"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.322822 3182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:26.342290 3182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:26.346851 3182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:26.347126 3182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:26.348096 3182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:26.349450 3182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:26.349527 3182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.349617 3182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:26.349665 3182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.385704 3182 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:36747
I20260807 18:21:26.385804 3369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:36747 every 8 connection(s)
I20260807 18:21:26.387105 3182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:26.392393 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3182
I20260807 18:21:26.392606 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:26.393043 30323 external_mini_cluster.cc:1428] Setting key 34a763cd536b1d6a37d7389c9705aca3
Aug 07 18:21:26 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:26.411386 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:26.390258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:52187 (local address 127.29.156.254:37677)
0807 18:21:26.390807 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:26.390812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:26.391751 (+ 939us) server_negotiation.cc:407] Connection header received
0807 18:21:26.393869 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:26.393875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:26.393963 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:26.394153 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:26.396165 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.397043 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.398751 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.399279 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.399427 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:26.403807 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:26.403812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:26.403816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:26.403856 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:26.407720 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:26.408536 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:26.408545 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:26.408551 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:26.408743 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:26.409194 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:26.409202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:26.409208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:26.409675 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:26.409835 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:26.410581 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 18:21:26.410851 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":335,"threads_started":1}
I20260807 18:21:26.412063 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:26.390783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37677 (local address 127.29.156.194:52187)
0807 18:21:26.391568 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:26.391623 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:26.393501 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:26.394419 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:26.394430 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:26.395302 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:26.395981 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:26.395999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.397207 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.397212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:26.398586 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:26.398600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.399569 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.399579 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:26.399839 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:26.400596 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:26.400641 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:26.403572 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:26.407869 (+ 4297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:26.407877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:26.407891 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:26.408380 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:26.408867 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:26.408871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:26.408873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:26.409061 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:26.409811 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:26.409817 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:26.410184 (+ 367us) client_negotiation.cc:769] Sending connection context
0807 18:21:26.410434 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 18:21:26.410866 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":277,"threads_started":1}
I20260807 18:21:26.413733 3371 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37677
I20260807 18:21:26.414099 3371 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:26.415120 3371 heartbeater.cc:511] Master 127.29.156.254:37677 requested a full tablet report, sending...
I20260807 18:21:26.417070 2925 ts_manager.cc:194] Registered new tserver with Master: 11d2be0faf4e44819f0f2bad55224749 (127.29.156.194:36747)
I20260807 18:21:26.417945 2925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:52187
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:26.455590 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:26.803411 3381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:26.803792 3381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:26.803898 3381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:26.811360 3381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:26.811479 3381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:26.811673 3381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:26.821890 3381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:26.823216 3381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:26.824621 3381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:26.838869 3387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:26.838819 3390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:26.838788 3392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:26.839788 3381 server_base.cc:1034] running on GCE node
I20260807 18:21:26.840476 3381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:26.841521 3381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:26.842753 3381 hybrid_clock.cc:648] HybridClock initialized: now 1786126886842693 us; error 55 us; skew 500 ppm
I20260807 18:21:26.843088 3381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:26 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:26.846555 3381 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:26.848286 3381 webserver.cc:460] Webserver started at http://127.29.156.195:39663/ using document root <none> and password file <none>
I20260807 18:21:26.849076 3381 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:26.849246 3381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:26.849608 3381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:26.853123 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "dbc0d39c36374df19321d60a8a72928b"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "26e31888f023085489bb1b34852a3820"
server_key_iv: "a78da724703364e1eadac739834d6d04"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.854054 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "dbc0d39c36374df19321d60a8a72928b"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "26e31888f023085489bb1b34852a3820"
server_key_iv: "a78da724703364e1eadac739834d6d04"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.860322 3381 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:21:26.864418 3401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.866117 3381 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:21:26.866356 3381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "dbc0d39c36374df19321d60a8a72928b"
format_stamp: "Formatted at 2026-08-07 18:21:26 on dist-test-slave-sb2z"
server_key: "26e31888f023085489bb1b34852a3820"
server_key_iv: "a78da724703364e1eadac739834d6d04"
server_key_version: "encryptionkey@0"
I20260807 18:21:26.866647 3381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:26.876873 3174 heartbeater.cc:503] Master 127.29.156.254:37677 was elected leader, sending a full tablet report...
I20260807 18:21:26.891793 3381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:26.897238 3381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:26.897585 3381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:26.898747 3381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:26.900346 3381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:26.900467 3381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.900653 3381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:26.900741 3381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:26.940136 3381 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:44097
I20260807 18:21:26.940318 3558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:44097 every 8 connection(s)
I20260807 18:21:26.941697 3381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:26.942907 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3381
I20260807 18:21:26.943115 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:26.943720 30323 external_mini_cluster.cc:1428] Setting key 0cc932a2da09227ea391311eaf00120a
Aug 07 18:21:26 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:26.965432 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:26.945027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:49031 (local address 127.29.156.254:37677)
0807 18:21:26.945555 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:26.945562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:26.946732 (+ 1170us) server_negotiation.cc:407] Connection header received
0807 18:21:26.949133 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:26.949138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:26.949215 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:26.949317 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:26.951059 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.951926 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.953616 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.954078 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.954182 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:26.958468 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:26.958473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:26.958477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:26.958519 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:26.962060 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:26.962970 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:26.962980 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:26.962982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:26.963145 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:26.963639 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:26.963643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:26.963645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:26.964001 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:26.964125 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:26.964916 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 18:21:26.965140 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":281,"threads_started":1}
I20260807 18:21:26.966382 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:26.945577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37677 (local address 127.29.156.195:49031)
0807 18:21:26.946536 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:26.946599 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:26.948828 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:26.949600 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:26.949614 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:26.950198 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:26.950851 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:26.950870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.952107 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.952112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:26.953461 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:26.953475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:26.954336 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:26.954342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:26.954596 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:26.955489 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:26.955520 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:26.958194 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:26.962216 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:26.962224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:26.962238 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:26.962784 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:26.963251 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:26.963254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:26.963257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:26.963465 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:26.964122 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:26.964128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:26.964486 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 18:21:26.964750 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 18:21:26.965170 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":363,"threads_started":1}
I20260807 18:21:26.968055 3559 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37677
I20260807 18:21:26.968400 3559 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:26.969195 3559 heartbeater.cc:511] Master 127.29.156.254:37677 requested a full tablet report, sending...
I20260807 18:21:26.970778 2925 ts_manager.cc:194] Registered new tserver with Master: dbc0d39c36374df19321d60a8a72928b (127.29.156.195:44097)
I20260807 18:21:26.972041 2925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:49031
I20260807 18:21:26.973412 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[2862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:27.038533 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:27.020987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.29.156.193:44475)
0807 18:21:27.021731 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:27.021739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:27.021765 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:21:27.021843 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:27.021850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:27.022089 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:27.022288 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:27.023943 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.025020 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.026406 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.027027 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.027197 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:27.031950 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:27.031964 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:27.031981 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:27.032064 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:27.035404 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.036229 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.036237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.036239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.036382 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.036822 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.036826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.036828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.037206 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:27.037559 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:27.037855 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:21:27.038158 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":350,"threads_started":1}
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:27.059664 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:27.044366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.29.156.254:37677)
0807 18:21:27.044618 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:27.044626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:27.044889 (+ 263us) server_negotiation.cc:407] Connection header received
0807 18:21:27.045142 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:27.045155 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:27.045223 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:27.045332 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:27.046627 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.047391 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.048687 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.049107 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.049250 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:27.053331 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:27.053337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:27.053340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:27.053378 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:27.056687 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.057396 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.057402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.057407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.057552 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.057941 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.057946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.057951 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.058311 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:27.058604 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:27.059003 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 18:21:27.059284 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 18:21:27.060273 2925 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37108
I20260807 18:21:27.061326 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3007
I20260807 18:21:27.075451 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:27.062330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.29.156.193:44475)
0807 18:21:27.062582 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:27.062589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:27.062741 (+ 152us) server_negotiation.cc:407] Connection header received
0807 18:21:27.062938 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:27.062943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:27.063047 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:27.063160 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:27.064496 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.065193 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.066430 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.066890 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.067040 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:27.069346 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:27.069350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:27.069354 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:27.069388 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:27.072539 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.073316 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.073320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.073323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.073466 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.073936 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.073941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.073944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.074256 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:27.074510 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:27.074793 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:21:27.075007 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 18:21:27.076290 3167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:27.169135 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3007
I20260807 18:21:27.195204 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3182
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:27.214166 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:27.196424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49432 (local address 127.29.156.194:36747)
0807 18:21:27.197258 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:27.197266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:27.197296 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:21:27.197371 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:27.197378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:27.197698 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:27.197860 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:27.199248 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.200548 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.202261 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.202884 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.203055 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:27.207831 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:27.207844 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:27.207861 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:27.207905 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:27.211182 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.211998 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.212006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.212008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.212151 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.212487 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.212490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.212492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.212870 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:27.213172 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:27.213575 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 18:21:27.213800 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":435,"threads_started":1}
I20260807 18:21:27.215727 3360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:27.303251 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3182
I20260807 18:21:27.334481 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3381
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[2862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:27.353323 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:27.335678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.29.156.195:44097)
0807 18:21:27.336418 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:27.336426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:27.336457 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:21:27.336521 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:27.336527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:27.336741 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:27.336883 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:27.338353 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.339927 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.341436 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:27.342081 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:27.342199 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:27.347117 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:27.347135 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:27.347153 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:27.347200 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:27.350423 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.351188 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.351195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.351198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.351335 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:27.351903 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:27.351907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:27.351909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:27.352290 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:27.352593 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:27.352809 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 18:21:27.353031 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":379,"threads_started":1}
I20260807 18:21:27.354797 3553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:27.439572 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3381
I20260807 18:21:27.463997 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 2881
I20260807 18:21:27.465394 2967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:27.576043 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 2881
2026-08-07T18:21:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4488 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[3585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[3585](info): set up 2 sockets
Aug 07 18:21:27 dist-test-slave-sb2z krb5kdc[3585](info): commencing operation
krb5kdc: starting...
W20260807 18:21:28.873049 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:29 dist-test-slave-sb2z krb5kdc[3585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:29.139479 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40057
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:41681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:29.484194 3609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:29.484505 3609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:29.484676 3609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:29.493518 3609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:29.493643 3609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:29.493896 3609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:29.494081 3609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:29.494170 3609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:29.504416 3609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:41681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:29.505800 3609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:29.507274 3609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:29.518942 3618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:29.519073 3616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:29.519286 3621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:29.520057 3609 server_base.cc:1034] running on GCE node
I20260807 18:21:29.520896 3609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:29.522293 3609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:29.523592 3609 hybrid_clock.cc:648] HybridClock initialized: now 1786126889523507 us; error 41 us; skew 500 ppm
I20260807 18:21:29.523952 3609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:29 dist-test-slave-sb2z krb5kdc[3585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:29.527235 3609 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:29.528479 3609 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:29.528620 3609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:29.528954 3609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:29.531981 3609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664"
format_stamp: "Formatted at 2026-08-07 18:21:29 on dist-test-slave-sb2z"
server_key: "8c864d92c004a7138640ea71cedb1f53"
server_key_iv: "19736d738043449028098714622fd3e0"
server_key_version: "encryptionkey@0"
I20260807 18:21:29.532812 3609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664"
format_stamp: "Formatted at 2026-08-07 18:21:29 on dist-test-slave-sb2z"
server_key: "8c864d92c004a7138640ea71cedb1f53"
server_key_iv: "19736d738043449028098714622fd3e0"
server_key_version: "encryptionkey@0"
I20260807 18:21:29.538169 3609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 18:21:29.541867 3635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:29.543190 3609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:21:29.543386 3609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664"
format_stamp: "Formatted at 2026-08-07 18:21:29 on dist-test-slave-sb2z"
server_key: "8c864d92c004a7138640ea71cedb1f53"
server_key_iv: "19736d738043449028098714622fd3e0"
server_key_version: "encryptionkey@0"
I20260807 18:21:29.543699 3609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:29.587471 3609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:29.592192 3609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:29.592489 3609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:29.612498 3609 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:40057
I20260807 18:21:29.612519 3703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:40057 every 8 connection(s)
I20260807 18:21:29.613507 3609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:29.617415 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:29.620263 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3609
I20260807 18:21:29.620522 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:29.621361 30323 external_mini_cluster.cc:1428] Setting key a6ac67b8ea2e8d39ac6ac05be4f13579
I20260807 18:21:29.629868 3705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: Bootstrap starting.
I20260807 18:21:29.632956 3705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:29.633939 3705 log.cc:824] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:21:29 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:29.637059 3705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: No bootstrap required, opened a new log
I20260807 18:21:29.645963 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:29.623725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.29.156.254:40057)
0807 18:21:29.624544 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:29.624555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:29.624596 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:21:29.626342 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:29.626365 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:29.626762 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:29.627182 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:29.628769 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:29.630099 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:29.631349 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:29.632114 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:29.632323 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:29.636700 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:29.636711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:29.636729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:29.636777 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:29.640958 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:29.641692 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:29.641701 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:29.641710 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:29.641933 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:29.642376 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:29.642381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:29.642386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:29.643160 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:29.643481 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:29.644050 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:21:29.644506 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":403,"threads_started":1}
I20260807 18:21:29.648075 3705 raft_consensus.cc:348] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } }
I20260807 18:21:29.648518 3705 raft_consensus.cc:374] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:29.648631 3705 raft_consensus.cc:729] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee1b26a78fb4e6bbf4b6a4fe7195664, State: Initialized, Role: FOLLOWER
I20260807 18:21:29.649246 3705 consensus_queue.cc:260] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } }
I20260807 18:21:29.649523 3705 raft_consensus.cc:388] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:29.649652 3705 raft_consensus.cc:482] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:29.649776 3705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:29.652014 3705 raft_consensus.cc:504] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } }
I20260807 18:21:29.652452 3705 leader_election.cc:302] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aee1b26a78fb4e6bbf4b6a4fe7195664; no voters:
I20260807 18:21:29.653192 3705 leader_election.cc:288] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:29.653326 3710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:29.654390 3710 raft_consensus.cc:686] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [term 1 LEADER]: Becoming Leader. State: Replica: aee1b26a78fb4e6bbf4b6a4fe7195664, State: Running, Role: LEADER
I20260807 18:21:29.654896 3710 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } }
I20260807 18:21:29.655437 3705 sys_catalog.cc:564] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:29.657667 3711 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } } }
I20260807 18:21:29.658028 3711 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:29.659663 3718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:29.659616 3712 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee1b26a78fb4e6bbf4b6a4fe7195664. Latest consensus state: current_term: 1 leader_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee1b26a78fb4e6bbf4b6a4fe7195664" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40057 } } }
I20260807 18:21:29.659957 3712 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:29.665334 3718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:29.675074 3718 catalog_manager.cc:1347] Generated new cluster ID: 7d29bf10e72248a79ab751dabba6d919
I20260807 18:21:29.675238 3718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:29.709072 3718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:29.710443 3718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:29.725449 3718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: Generated new TSK 0
I20260807 18:21:29.726297 3718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:29.800262 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:41681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:30.153487 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:30.153782 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:30.153962 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:30.162168 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:30.162302 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:30.162541 3744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:30.173148 3744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:41681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:30.174564 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:30.176057 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:30.190089 3751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:30.190017 3753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:30.190239 3756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:30.191049 3744 server_base.cc:1034] running on GCE node
I20260807 18:21:30.191749 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:30.192744 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:30.194021 3744 hybrid_clock.cc:648] HybridClock initialized: now 1786126890193893 us; error 64 us; skew 500 ppm
I20260807 18:21:30.194347 3744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:30 dist-test-slave-sb2z krb5kdc[3585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:30.201687 3744 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:30.203159 3744 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:30.203281 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:30.203769 3744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:30.207118 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "faecdabee84e4fc6b3482356e1bf87c8"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "d8d7b1199beb5d51e7255485468418d1"
server_key_iv: "ad0aa45a041132015cfffb77634cc504"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.208022 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "faecdabee84e4fc6b3482356e1bf87c8"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "d8d7b1199beb5d51e7255485468418d1"
server_key_iv: "ad0aa45a041132015cfffb77634cc504"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.213598 3744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 18:21:30.217097 3767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.218555 3744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:21:30.218737 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "faecdabee84e4fc6b3482356e1bf87c8"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "d8d7b1199beb5d51e7255485468418d1"
server_key_iv: "ad0aa45a041132015cfffb77634cc504"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.218961 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:30.234526 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:30.239356 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:30.239713 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:30.240744 3744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:30.242156 3744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:30.242273 3744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.242412 3744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:30.242516 3744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.280296 3744 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:41401
I20260807 18:21:30.280416 3922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:41401 every 8 connection(s)
I20260807 18:21:30.281288 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:30.290310 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3744
I20260807 18:21:30.290575 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:30.291337 30323 external_mini_cluster.cc:1428] Setting key f2fd9b33b1c1777bcd0f7eaf6cae32fb
Aug 07 18:21:30 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:30.305274 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:30.284038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:44915 (local address 127.29.156.254:40057)
0807 18:21:30.284569 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:30.284574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:30.285398 (+ 824us) server_negotiation.cc:407] Connection header received
0807 18:21:30.287456 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:30.287464 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:30.287602 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:30.287738 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:30.289390 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.290506 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.292426 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.292951 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.293072 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:30.297203 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:30.297214 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:30.297222 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:30.297275 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:30.301362 (+ 4087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:30.302247 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:30.302256 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:30.302263 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:30.302428 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:30.302878 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:30.302885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:30.302892 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:30.303344 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:30.303515 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:30.304484 (+ 969us) server_negotiation.cc:299] Negotiation successful
0807 18:21:30.304778 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":275,"threads_started":1}
I20260807 18:21:30.306052 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:30.284437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40057 (local address 127.29.156.193:44915)
0807 18:21:30.285213 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:30.285261 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:30.287164 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:30.287984 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:30.287993 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:30.288590 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:30.289195 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:30.289219 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.290739 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.290744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:30.292269 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:30.292284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.293215 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.293223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:30.293493 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:30.294255 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:30.294279 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:30.296918 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:30.301560 (+ 4642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:30.301568 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:30.301584 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:30.302105 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:30.302554 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:30.302558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:30.302560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:30.302759 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:30.303594 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:30.303603 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:30.304053 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 18:21:30.304310 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 18:21:30.304765 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":327,"threads_started":1}
I20260807 18:21:30.307768 3926 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40057
I20260807 18:21:30.308166 3926 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:30.309010 3926 heartbeater.cc:511] Master 127.29.156.254:40057 requested a full tablet report, sending...
I20260807 18:21:30.311398 3654 ts_manager.cc:194] Registered new tserver with Master: faecdabee84e4fc6b3482356e1bf87c8 (127.29.156.193:41401)
I20260807 18:21:30.313056 3654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:44915
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:30.351639 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:41681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:30.705042 3937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:30.705367 3937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:30.705550 3937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:30.713274 3937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:30.713392 3937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:30.713604 3937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:30.723836 3937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:41681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:30.725277 3937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:30.726796 3937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:30.740804 3946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:30.740761 3948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:30.740911 3951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:30.742025 3937 server_base.cc:1034] running on GCE node
I20260807 18:21:30.742713 3937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:30.743790 3937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:30.744998 3937 hybrid_clock.cc:648] HybridClock initialized: now 1786126890744956 us; error 41 us; skew 500 ppm
I20260807 18:21:30.745352 3937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:30 dist-test-slave-sb2z krb5kdc[3585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:30.749063 3937 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:30.750231 3937 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:30.750326 3937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:30.750602 3937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:30.753665 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "1138777be3234038b471e9c626851846"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "fc06d337c31e2d6ac889d7265ffddc9b"
server_key_iv: "39054d403ecac975f443106ccfcaa5bc"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.754472 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "1138777be3234038b471e9c626851846"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "fc06d337c31e2d6ac889d7265ffddc9b"
server_key_iv: "39054d403ecac975f443106ccfcaa5bc"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.760195 3937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:21:30.764055 3961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.765604 3937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:30.765763 3937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "1138777be3234038b471e9c626851846"
format_stamp: "Formatted at 2026-08-07 18:21:30 on dist-test-slave-sb2z"
server_key: "fc06d337c31e2d6ac889d7265ffddc9b"
server_key_iv: "39054d403ecac975f443106ccfcaa5bc"
server_key_version: "encryptionkey@0"
I20260807 18:21:30.765954 3937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:30.782480 3937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:30.787617 3937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:30.787914 3937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:30.789085 3937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:30.790571 3937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:30.790673 3937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.790758 3937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:30.790824 3937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:30.829805 3937 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:39017
I20260807 18:21:30.829913 4109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:39017 every 8 connection(s)
I20260807 18:21:30.830808 3937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:30.840008 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 3937
I20260807 18:21:30.840219 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:30.840735 30323 external_mini_cluster.cc:1428] Setting key d62cf91de9340740e2a3fd0c75d7f6b1
Aug 07 18:21:30 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:30.855620 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:30.833696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:58143 (local address 127.29.156.254:40057)
0807 18:21:30.834245 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:30.834250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:30.835158 (+ 908us) server_negotiation.cc:407] Connection header received
0807 18:21:30.837288 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:30.837296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:30.837408 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:30.837545 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:30.839295 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.840229 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.842408 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.843148 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.843302 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:30.847617 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:30.847628 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:30.847636 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:30.847688 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:30.851818 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:30.852794 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:30.852803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:30.852810 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:30.852988 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:30.853485 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:30.853493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:30.853500 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:30.853913 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:30.854100 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:30.854874 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 18:21:30.855132 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":302,"threads_started":1}
I20260807 18:21:30.856750 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:30.834111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40057 (local address 127.29.156.194:58143)
0807 18:21:30.834963 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:30.835011 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:30.836972 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:30.837809 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:30.837819 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:30.838432 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:30.839074 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:30.839099 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.840450 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.840455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:30.842241 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:30.842258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:30.843401 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:30.843406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:30.843730 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:30.844542 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:30.844566 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:30.847278 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:30.852003 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:30.852014 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:30.852035 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:30.852652 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:30.853113 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:30.853120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:30.853126 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:30.853359 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:30.854084 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:30.854093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:30.854476 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 18:21:30.854786 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 18:21:30.855440 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":320,"threads_started":1}
I20260807 18:21:30.858489 4110 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40057
I20260807 18:21:30.858847 4110 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:30.859669 4110 heartbeater.cc:511] Master 127.29.156.254:40057 requested a full tablet report, sending...
I20260807 18:21:30.861392 3654 ts_manager.cc:194] Registered new tserver with Master: 1138777be3234038b471e9c626851846 (127.29.156.194:39017)
I20260807 18:21:30.862324 3654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:58143
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:30.903381 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:41681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:31.260043 4117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:31.260329 4117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:31.260432 4117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:31.268122 4117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:31.268239 4117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:31.268360 4117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:31.278810 4117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:41681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:31.280620 4117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:31.282446 4117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:31.298094 4122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:31.298056 4123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:31.298125 4126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:31.298780 4117 server_base.cc:1034] running on GCE node
I20260807 18:21:31.299415 4117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:31.300438 4117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:31.301661 4117 hybrid_clock.cc:648] HybridClock initialized: now 1786126891301615 us; error 29 us; skew 500 ppm
I20260807 18:21:31.301991 4117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:31 dist-test-slave-sb2z krb5kdc[3585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126891, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:31.305406 4117 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:31.306555 4117 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:31.306650 4117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:31.306917 4117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:31.310040 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "491be5d162884d0185b0c11e3573fc0e"
format_stamp: "Formatted at 2026-08-07 18:21:31 on dist-test-slave-sb2z"
server_key: "c8d8bdf0bcb6a10ab094459c80ff041d"
server_key_iv: "f7b71a1e786b39f1a35cf34f4f943f9b"
server_key_version: "encryptionkey@0"
I20260807 18:21:31.310842 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "491be5d162884d0185b0c11e3573fc0e"
format_stamp: "Formatted at 2026-08-07 18:21:31 on dist-test-slave-sb2z"
server_key: "c8d8bdf0bcb6a10ab094459c80ff041d"
server_key_iv: "f7b71a1e786b39f1a35cf34f4f943f9b"
server_key_version: "encryptionkey@0"
I20260807 18:21:31.316133 4117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 18:21:31.316573 3926 heartbeater.cc:503] Master 127.29.156.254:40057 was elected leader, sending a full tablet report...
I20260807 18:21:31.321998 4137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:31.323917 4117 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:21:31.324182 4117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "491be5d162884d0185b0c11e3573fc0e"
format_stamp: "Formatted at 2026-08-07 18:21:31 on dist-test-slave-sb2z"
server_key: "c8d8bdf0bcb6a10ab094459c80ff041d"
server_key_iv: "f7b71a1e786b39f1a35cf34f4f943f9b"
server_key_version: "encryptionkey@0"
I20260807 18:21:31.324501 4117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:31.341142 4117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:31.346101 4117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:31.346443 4117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:31.347728 4117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:31.349436 4117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:31.349577 4117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:31.349732 4117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:31.349838 4117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:31.388813 4117 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45323
I20260807 18:21:31.388864 4294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45323 every 8 connection(s)
I20260807 18:21:31.389791 4117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:31.390762 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 4117
I20260807 18:21:31.390983 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:31.391564 30323 external_mini_cluster.cc:1428] Setting key e2f297da969c8b209abe6fb6aad52e37
Aug 07 18:21:31 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126891, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:31.412510 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.392988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:49921 (local address 127.29.156.254:40057)
0807 18:21:31.395803 (+ 2815us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.395808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.395826 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:21:31.397156 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.397160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.397225 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.397321 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:31.399048 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.399901 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.401720 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.402133 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.402232 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.406193 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:31.406199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:31.406202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:31.406240 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:31.409386 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.410168 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.410172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.410174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.410298 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.410714 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.410717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.410719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.410974 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:31.411133 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.412040 (+ 907us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.412247 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2705,"thread_start_us":2560,"threads_started":1}
I20260807 18:21:31.413715 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.393768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40057 (local address 127.29.156.195:49921)
0807 18:21:31.394811 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.394872 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.396875 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.397632 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.397642 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.398256 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:31.398864 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.398883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.400066 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.400071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.401575 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.401589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.402373 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.402386 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.402608 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.403314 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:31.403342 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:31.405893 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:31.409524 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:31.409531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:31.409545 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:31.410011 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:31.410408 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:31.410412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:31.410414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:31.410614 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:31.411131 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:31.411138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:31.411508 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.411958 (+ 450us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.412441 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":428,"threads_started":1}
I20260807 18:21:31.415346 4295 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40057
I20260807 18:21:31.415728 4295 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:31.416379 4295 heartbeater.cc:511] Master 127.29.156.254:40057 requested a full tablet report, sending...
I20260807 18:21:31.417829 3654 ts_manager.cc:194] Registered new tserver with Master: 491be5d162884d0185b0c11e3573fc0e (127.29.156.195:45323)
I20260807 18:21:31.419050 3654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:49921
I20260807 18:21:31.420553 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:21:31.438125 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.425312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.29.156.254:40057)
0807 18:21:31.425534 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.425543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.426004 (+ 461us) server_negotiation.cc:407] Connection header received
0807 18:21:31.426165 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.426168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.426230 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.426312 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:31.427839 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.428640 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.429995 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.430416 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.430516 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.432835 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:31.432840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:31.432843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:31.432875 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:31.435599 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.436281 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.436285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.436287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.436401 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.436778 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.436781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.436783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.437078 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:31.437267 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.437570 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.437740 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:21:31.442641 3654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:31.449549 3654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:31.484884 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.463983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.29.156.194:39017)
0807 18:21:31.471991 (+ 8008us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.472005 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.472058 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:21:31.472232 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.472246 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.472655 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.475385 (+ 2730us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:31.475785 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.478136 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.482003 (+ 3867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483356 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.483625 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.483864 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.484275 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.484477 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7413,"thread_start_us":1267,"threads_started":1}
I20260807 18:21:31.484972 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.463264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39017 (local address 127.0.0.1:51152)
0807 18:21:31.464394 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.464454 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.464642 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.474103 (+ 9461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.474116 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.474173 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:31.474800 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.474824 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.478419 (+ 3595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.478430 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.481693 (+ 3263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.481722 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483626 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.483638 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.483913 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.483994 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.484179 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.484447 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":69,"thread_start_us":14435,"threads_started":1}
I20260807 18:21:31.489181 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.463980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.29.156.195:45323)
0807 18:21:31.479243 (+ 15263us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.479250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.479278 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:21:31.479357 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.479363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.479643 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.479877 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:31.480991 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483247 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.486715 (+ 3468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.487907 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.488148 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.488341 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.488630 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.488805 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15079,"thread_start_us":16910,"threads_started":1}
I20260807 18:21:31.489241 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.463486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45323 (local address 127.0.0.1:57798)
0807 18:21:31.477364 (+ 13878us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.477400 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.477592 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.480104 (+ 2512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.480110 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.480147 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:31.480702 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.480716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483409 (+ 2693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.483418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.486509 (+ 3091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.486530 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.488148 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.488156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.488337 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.488384 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.488540 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.488725 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13686,"mutex_wait_us":1,"spinlock_wait_cycles":22272,"thread_start_us":742,"threads_started":1}
I20260807 18:21:31.489429 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.465120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:41401 (local address 127.0.0.1:46390)
0807 18:21:31.480158 (+ 15038us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.480187 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.480380 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.481268 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.481273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.481297 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:31.481844 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.481861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483833 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.483838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.486226 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.486241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.488519 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.488525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.488741 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.488776 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.488869 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.488998 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14894,"mutex_wait_us":65,"spinlock_wait_cycles":21120,"thread_start_us":14710,"threads_started":1}
I20260807 18:21:31.490178 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.465436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.29.156.193:41401)
0807 18:21:31.475158 (+ 9722us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.475178 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.480316 (+ 5138us) server_negotiation.cc:407] Connection header received
0807 18:21:31.480565 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.480571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.480751 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.480978 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:31.482078 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.483664 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.486423 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.487977 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.488174 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.488335 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.489663 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.489839 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9485,"thread_start_us":9487,"threads_started":1}
I20260807 18:21:31.489856 4024 tablet_service.cc:1467] Processing CreateTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 (DEFAULT_TABLE table=test-table [id=2228592d64524e75a225a9c5de2b75cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:31.491587 4024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f69e3d477954a8f86b8e229d5e15c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:31.493333 4203 tablet_service.cc:1467] Processing CreateTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 (DEFAULT_TABLE table=test-table [id=2228592d64524e75a225a9c5de2b75cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:31.494637 4203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f69e3d477954a8f86b8e229d5e15c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:31.495067 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 (DEFAULT_TABLE table=test-table [id=2228592d64524e75a225a9c5de2b75cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:31.497907 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f69e3d477954a8f86b8e229d5e15c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:31.505977 4323 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Bootstrap starting.
I20260807 18:21:31.509634 4323 tablet_bootstrap.cc:654] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:31.510682 4323 log.cc:824] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:31.511974 4324 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Bootstrap starting.
I20260807 18:21:31.515734 4323 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: No bootstrap required, opened a new log
I20260807 18:21:31.515811 4324 tablet_bootstrap.cc:654] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:31.516016 4323 ts_tablet_manager.cc:1397] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260807 18:21:31.516256 4326 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Bootstrap starting.
I20260807 18:21:31.518159 4324 log.cc:824] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:31.520031 4326 tablet_bootstrap.cc:654] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:31.521456 4326 log.cc:824] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:31.521802 4324 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: No bootstrap required, opened a new log
I20260807 18:21:31.522269 4324 ts_tablet_manager.cc:1397] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:21:31.525019 4326 tablet_bootstrap.cc:492] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: No bootstrap required, opened a new log
I20260807 18:21:31.525012 4323 raft_consensus.cc:348] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.525326 4326 ts_tablet_manager.cc:1397] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:21:31.525458 4323 raft_consensus.cc:374] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:31.525575 4323 raft_consensus.cc:729] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1138777be3234038b471e9c626851846, State: Initialized, Role: FOLLOWER
I20260807 18:21:31.526226 4323 consensus_queue.cc:260] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.527346 4110 heartbeater.cc:503] Master 127.29.156.254:40057 was elected leader, sending a full tablet report...
I20260807 18:21:31.530123 4323 ts_tablet_manager.cc:1428] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Time spent starting tablet: real 0.014s user 0.006s sys 0.005s
I20260807 18:21:31.534586 4324 raft_consensus.cc:348] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.535110 4324 raft_consensus.cc:374] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:31.535228 4324 raft_consensus.cc:729] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491be5d162884d0185b0c11e3573fc0e, State: Initialized, Role: FOLLOWER
I20260807 18:21:31.536007 4324 consensus_queue.cc:260] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.537216 4295 heartbeater.cc:503] Master 127.29.156.254:40057 was elected leader, sending a full tablet report...
I20260807 18:21:31.537724 4326 raft_consensus.cc:348] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.538187 4324 ts_tablet_manager.cc:1428] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260807 18:21:31.538234 4326 raft_consensus.cc:374] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:31.538347 4326 raft_consensus.cc:729] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faecdabee84e4fc6b3482356e1bf87c8, State: Initialized, Role: FOLLOWER
I20260807 18:21:31.539013 4326 consensus_queue.cc:260] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.540903 4326 ts_tablet_manager.cc:1428] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
W20260807 18:21:31.582988 4111 tablet.cc:2369] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:21:31.641657 4296 tablet.cc:2369] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:31.746019 4335 raft_consensus.cc:482] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:31.746366 4335 raft_consensus.cc:504] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.748358 4335 leader_election.cc:288] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1138777be3234038b471e9c626851846 (127.29.156.194:39017), 491be5d162884d0185b0c11e3573fc0e (127.29.156.195:45323)
I20260807 18:21:31.758526 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.748956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:53019 (local address 127.29.156.195:45323)
0807 18:21:31.749204 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.749213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.749926 (+ 713us) server_negotiation.cc:407] Connection header received
0807 18:21:31.750135 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.750141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.750221 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.750330 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:31.751366 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.752589 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.755874 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.757273 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.757464 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.757544 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.758085 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.758232 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:21:31.758525 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.748970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38743 (local address 127.29.156.194:39017)
0807 18:21:31.749264 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.749294 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.749935 (+ 641us) server_negotiation.cc:407] Connection header received
0807 18:21:31.750112 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.750118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.750211 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.750330 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:31.751366 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.752619 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.755868 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.757293 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.757454 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.757544 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.758084 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.758232 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:21:31.758697 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.748733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45323 (local address 127.29.156.193:53019)
0807 18:21:31.749749 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.749771 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.749968 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.750361 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.750366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.750401 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:31.751139 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.751154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.752755 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.752760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.755688 (+ 2928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.755706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.757581 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.757591 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.757820 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.757876 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.758002 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.758162 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":222,"spinlock_wait_cycles":18432,"thread_start_us":517,"threads_started":1}
I20260807 18:21:31.759233 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.748731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39017 (local address 127.29.156.193:38743)
0807 18:21:31.749816 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:31.749834 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:31.749968 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:31.750361 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:31.750366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:31.750403 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:31.751145 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.751154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.752756 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.752760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.755688 (+ 2928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:31.755706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.757631 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.757637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:31.757814 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.757880 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 18:21:31.757990 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:21:31.758162 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":346,"spinlock_wait_cycles":27008,"thread_start_us":743,"threads_started":1}
I20260807 18:21:31.759889 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f69e3d477954a8f86b8e229d5e15c49" candidate_uuid: "faecdabee84e4fc6b3482356e1bf87c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491be5d162884d0185b0c11e3573fc0e" is_pre_election: true
I20260807 18:21:31.760386 4049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f69e3d477954a8f86b8e229d5e15c49" candidate_uuid: "faecdabee84e4fc6b3482356e1bf87c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138777be3234038b471e9c626851846" is_pre_election: true
I20260807 18:21:31.760550 4229 raft_consensus.cc:2454] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faecdabee84e4fc6b3482356e1bf87c8 in term 0.
I20260807 18:21:31.760988 4049 raft_consensus.cc:2454] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faecdabee84e4fc6b3482356e1bf87c8 in term 0.
I20260807 18:21:31.761340 3771 leader_election.cc:302] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491be5d162884d0185b0c11e3573fc0e, faecdabee84e4fc6b3482356e1bf87c8; no voters:
I20260807 18:21:31.761809 4335 raft_consensus.cc:2781] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:31.761978 4335 raft_consensus.cc:482] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:31.762164 4335 raft_consensus.cc:3037] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:31.764635 4335 raft_consensus.cc:504] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.765275 4335 leader_election.cc:288] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [CANDIDATE]: Term 1 election: Requested vote from peers 1138777be3234038b471e9c626851846 (127.29.156.194:39017), 491be5d162884d0185b0c11e3573fc0e (127.29.156.195:45323)
I20260807 18:21:31.765753 4049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f69e3d477954a8f86b8e229d5e15c49" candidate_uuid: "faecdabee84e4fc6b3482356e1bf87c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138777be3234038b471e9c626851846"
I20260807 18:21:31.765812 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f69e3d477954a8f86b8e229d5e15c49" candidate_uuid: "faecdabee84e4fc6b3482356e1bf87c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491be5d162884d0185b0c11e3573fc0e"
I20260807 18:21:31.765945 4049 raft_consensus.cc:3037] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:31.766052 4229 raft_consensus.cc:3037] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:31.768564 4229 raft_consensus.cc:2454] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faecdabee84e4fc6b3482356e1bf87c8 in term 1.
I20260807 18:21:31.768584 4049 raft_consensus.cc:2454] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faecdabee84e4fc6b3482356e1bf87c8 in term 1.
I20260807 18:21:31.768980 3771 leader_election.cc:302] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491be5d162884d0185b0c11e3573fc0e, faecdabee84e4fc6b3482356e1bf87c8; no voters:
I20260807 18:21:31.769332 4335 raft_consensus.cc:2781] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:31.769785 4335 raft_consensus.cc:686] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 1 LEADER]: Becoming Leader. State: Replica: faecdabee84e4fc6b3482356e1bf87c8, State: Running, Role: LEADER
I20260807 18:21:31.770309 4335 consensus_queue.cc:237] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } }
I20260807 18:21:31.775597 3654 catalog_manager.cc:5495] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to faecdabee84e4fc6b3482356e1bf87c8 (127.29.156.193). New cstate: current_term: 1 leader_uuid: "faecdabee84e4fc6b3482356e1bf87c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faecdabee84e4fc6b3482356e1bf87c8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41401 } health_report { overall_health: HEALTHY } } }
W20260807 18:21:31.783886 3927 tablet.cc:2369] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:31.860347 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.854307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.29.156.193:41401)
0807 18:21:31.854582 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.854590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.854729 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:21:31.854994 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.854998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.855084 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.855257 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:31.855736 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.856507 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.858032 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.858452 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.858602 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.859132 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:31.859741 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:31.859905 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.859966 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.860065 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:21:31.869515 4229 raft_consensus.cc:1261] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 1 FOLLOWER]: Refusing update from remote peer faecdabee84e4fc6b3482356e1bf87c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:31.869515 4049 raft_consensus.cc:1261] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 1 FOLLOWER]: Refusing update from remote peer faecdabee84e4fc6b3482356e1bf87c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:31.870813 4338 consensus_queue.cc:1035] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1138777be3234038b471e9c626851846" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:31.871302 4335 consensus_queue.cc:1035] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "491be5d162884d0185b0c11e3573fc0e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:31.883162 4342 mvcc.cc:205] Tried to move back new op lower bound from 7315975749084827648 to 7315975748695142400. Current Snapshot: MvccSnapshot[applied={T|T < 7315975749084827648}]
I20260807 18:21:31.898473 3652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:31.898793 3652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:31.902931 3652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aee1b26a78fb4e6bbf4b6a4fe7195664: Sending DeleteTablet for 3 replicas of tablet 7f69e3d477954a8f86b8e229d5e15c49
I20260807 18:21:31.904230 4024 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:51152
I20260807 18:21:31.904358 3834 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46390
I20260807 18:21:31.904341 4203 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f69e3d477954a8f86b8e229d5e15c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:57798
I20260807 18:21:31.907130 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3744
I20260807 18:21:31.919975 4360 tablet_replica.cc:330] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: stopping tablet replica
I20260807 18:21:31.920533 4360 raft_consensus.cc:2229] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:31.921168 4360 raft_consensus.cc:2258] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:31.922981 4358 tablet_replica.cc:330] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: stopping tablet replica
I20260807 18:21:31.923465 4358 raft_consensus.cc:2229] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:31.923974 4358 raft_consensus.cc:2258] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:31.925007 4359 tablet_replica.cc:330] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: stopping tablet replica
I20260807 18:21:31.925436 4359 raft_consensus.cc:2229] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:31.926095 4359 raft_consensus.cc:2258] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:31.926278 4358 ts_tablet_manager.cc:1905] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:31.927839 4360 ts_tablet_manager.cc:1905] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:31.929943 4359 ts_tablet_manager.cc:1905] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:31.933961 4358 ts_tablet_manager.cc:1918] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:31.934185 4358 log.cc:1196] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/7f69e3d477954a8f86b8e229d5e15c49
I20260807 18:21:31.934875 4358 ts_tablet_manager.cc:1939] T 7f69e3d477954a8f86b8e229d5e15c49 P 1138777be3234038b471e9c626851846: Deleting consensus metadata
Aug 07 18:21:31 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:31.937592 4359 ts_tablet_manager.cc:1918] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:31.937554 3638 catalog_manager.cc:4841] TS 1138777be3234038b471e9c626851846 (127.29.156.194:39017): tablet 7f69e3d477954a8f86b8e229d5e15c49 (table test-table [id=2228592d64524e75a225a9c5de2b75cd]) successfully deleted
I20260807 18:21:31.937783 4359 log.cc:1196] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/7f69e3d477954a8f86b8e229d5e15c49
I20260807 18:21:31.939186 4359 ts_tablet_manager.cc:1939] T 7f69e3d477954a8f86b8e229d5e15c49 P faecdabee84e4fc6b3482356e1bf87c8: Deleting consensus metadata
I20260807 18:21:31.941149 3639 catalog_manager.cc:4841] TS faecdabee84e4fc6b3482356e1bf87c8 (127.29.156.193:41401): tablet 7f69e3d477954a8f86b8e229d5e15c49 (table test-table [id=2228592d64524e75a225a9c5de2b75cd]) successfully deleted
I20260807 18:21:31.944478 4360 ts_tablet_manager.cc:1918] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:31.944674 4360 log.cc:1196] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/7f69e3d477954a8f86b8e229d5e15c49
I20260807 18:21:31.945098 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:31.908446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.29.156.193:41401)
0807 18:21:31.908634 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:31.908640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:31.922298 (+ 13658us) server_negotiation.cc:407] Connection header received
0807 18:21:31.922518 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:31.922528 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:31.922674 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:31.922818 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:31.926128 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.927370 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.929630 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:31.930365 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:31.930473 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:31.936454 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:31.936468 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:31.936486 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:31.936539 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:31.941893 (+ 5354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.942643 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.942650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.942653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.942819 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:31.943220 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:31.943230 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:31.943232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:31.943652 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:31.943961 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:31.944255 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:21:31.944507 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:21:31.945204 4360 ts_tablet_manager.cc:1939] T 7f69e3d477954a8f86b8e229d5e15c49 P 491be5d162884d0185b0c11e3573fc0e: Deleting consensus metadata
I20260807 18:21:31.945778 3916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:31.947202 3639 catalog_manager.cc:4841] TS 491be5d162884d0185b0c11e3573fc0e (127.29.156.195:45323): tablet 7f69e3d477954a8f86b8e229d5e15c49 (table test-table [id=2228592d64524e75a225a9c5de2b75cd]) successfully deleted
I20260807 18:21:32.047597 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3744
I20260807 18:21:32.075028 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3937
Aug 07 18:21:32 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:32.091157 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:32.075920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51164 (local address 127.29.156.194:39017)
0807 18:21:32.076193 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:32.076201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:32.076313 (+ 112us) server_negotiation.cc:407] Connection header received
0807 18:21:32.076551 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:32.076557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:32.076643 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:32.076754 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:32.078061 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:32.078812 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:32.080199 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:32.080620 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:32.080766 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:32.085035 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:32.085051 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:32.085067 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:32.085100 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:32.088227 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:32.088897 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:32.088907 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:32.088911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:32.089066 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:32.089493 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:32.089499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:32.089504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:32.089923 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:32.090238 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:32.090551 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 18:21:32.090779 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 18:21:32.091905 4104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:32.183182 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3937
I20260807 18:21:32.205803 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 4117
Aug 07 18:21:32 dist-test-slave-sb2z krb5kdc[3585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:32.221808 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:32.206659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.29.156.195:45323)
0807 18:21:32.206846 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:32.206851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:32.206871 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:21:32.206998 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:32.207003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:32.207067 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:32.207207 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:32.208548 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:32.209333 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:32.210600 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:32.211014 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:32.211145 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:32.215698 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:32.215723 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:32.215750 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:32.215793 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:32.218679 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:32.219357 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:32.219365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:32.219368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:32.219611 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:32.220089 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:32.220096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:32.220098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:32.220529 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:32.220795 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:32.221230 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:21:32.221405 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:21:32.222502 4290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:32.312625 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 4117
I20260807 18:21:32.337242 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 3609
I20260807 18:21:32.338464 3698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:32.450238 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 3609
2026-08-07T18:21:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4873 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:21:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:32Z Disabled control of system clock
I20260807 18:21:32.540323 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:35073
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:35073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:21:32.887838 4374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:32.888162 4374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:32.888303 4374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:32.895897 4374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:32.896010 4374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:32.896056 4374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:32.896148 4374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:32.896211 4374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:32.905906 4374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:35073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:35073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:32.907335 4374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:32.909219 4374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:32.922013 4383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:32.922181 4382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:32.922156 4385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:32.923848 4374 server_base.cc:1034] running on GCE node
I20260807 18:21:32.924783 4374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:32.926474 4374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:32.927687 4374 hybrid_clock.cc:648] HybridClock initialized: now 1786126892927609 us; error 64 us; skew 500 ppm
I20260807 18:21:32.928182 4374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:32.931039 4374 webserver.cc:460] Webserver started at http://127.29.156.254:39927/ using document root <none> and password file <none>
I20260807 18:21:32.931957 4374 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:32.932070 4374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:32.932451 4374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:32.935568 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "1b4aaeafb6564d27be991ec389795c74"
format_stamp: "Formatted at 2026-08-07 18:21:32 on dist-test-slave-sb2z"
server_key: "90ac617d1705b53532db9e3effad9258"
server_key_iv: "49f87f5cd06b973242b770c9dbcc944f"
server_key_version: "encryptionkey@0"
I20260807 18:21:32.936437 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "1b4aaeafb6564d27be991ec389795c74"
format_stamp: "Formatted at 2026-08-07 18:21:32 on dist-test-slave-sb2z"
server_key: "90ac617d1705b53532db9e3effad9258"
server_key_iv: "49f87f5cd06b973242b770c9dbcc944f"
server_key_version: "encryptionkey@0"
I20260807 18:21:32.941921 4374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:21:32.945791 4396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:32.947448 4374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:32.947744 4374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "1b4aaeafb6564d27be991ec389795c74"
format_stamp: "Formatted at 2026-08-07 18:21:32 on dist-test-slave-sb2z"
server_key: "90ac617d1705b53532db9e3effad9258"
server_key_iv: "49f87f5cd06b973242b770c9dbcc944f"
server_key_version: "encryptionkey@0"
I20260807 18:21:32.948006 4374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:32.954660 4374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:32.955688 4374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:32.956039 4374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:32.978057 4374 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:35073
I20260807 18:21:32.978163 4468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:35073 every 8 connection(s)
I20260807 18:21:32.979652 4374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:32.983860 4469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:32.986879 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 4374
I20260807 18:21:32.987252 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:32.987995 30323 external_mini_cluster.cc:1428] Setting key ba864b573d2f9f1f18f1b414d587b872
I20260807 18:21:32.995779 4469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: Bootstrap starting.
I20260807 18:21:32.998275 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:32.990453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.29.156.254:35073)
0807 18:21:32.991724 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:32.991737 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:32.991787 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:21:32.994289 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:32.994316 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:32.994796 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:32.995177 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:32.996512 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:32.996518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:32.996532 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:32.996548 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:32.996620 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:32.996637 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:32.996824 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:32.996936 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 18:21:32.997255 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":449,"threads_started":1}
I20260807 18:21:32.999743 4469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:33.000909 4469 log.cc:824] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:33.003867 4469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: No bootstrap required, opened a new log
I20260807 18:21:33.012151 4469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } }
I20260807 18:21:33.012584 4469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:33.012699 4469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4aaeafb6564d27be991ec389795c74, State: Initialized, Role: FOLLOWER
I20260807 18:21:33.013291 4469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } }
I20260807 18:21:33.013552 4469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:33.013705 4469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:33.013893 4469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:33.016777 4469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } }
I20260807 18:21:33.017495 4469 leader_election.cc:302] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b4aaeafb6564d27be991ec389795c74; no voters:
I20260807 18:21:33.018368 4469 leader_election.cc:288] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:33.018656 4475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:33.019791 4475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [term 1 LEADER]: Becoming Leader. State: Replica: 1b4aaeafb6564d27be991ec389795c74, State: Running, Role: LEADER
I20260807 18:21:33.020519 4475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } }
I20260807 18:21:33.026135 4469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:33.027045 4478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b4aaeafb6564d27be991ec389795c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } } }
I20260807 18:21:33.027489 4478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:33.029137 4477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4aaeafb6564d27be991ec389795c74. Latest consensus state: current_term: 1 leader_uuid: "1b4aaeafb6564d27be991ec389795c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4aaeafb6564d27be991ec389795c74" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35073 } } }
I20260807 18:21:33.029410 4477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:33.029593 4482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:33.037360 4482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:33.049093 4482 catalog_manager.cc:1347] Generated new cluster ID: 6dc172dd1aac4f1b92b7bd441b76074c
I20260807 18:21:33.049222 4482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:33.094106 4482 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:33.094600 4482 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:33.111609 4482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: Generated new TSK 0
I20260807 18:21:33.112586 4482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:21:33.132072 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--builtin_ntp_servers=127.29.156.212:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:21:33.483927 4506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:33.484236 4506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:33.484390 4506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:33.491956 4506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:33.492070 4506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:33.492225 4506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:33.502813 4506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:33.504783 4506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:33.507016 4506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:33.521673 4514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:33.521737 4515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:33.522413 4517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:33.522949 4506 server_base.cc:1034] running on GCE node
I20260807 18:21:33.523675 4506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:33.524698 4506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:33.525966 4506 hybrid_clock.cc:648] HybridClock initialized: now 1786126893525914 us; error 40 us; skew 500 ppm
I20260807 18:21:33.526355 4506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:33.529253 4506 webserver.cc:460] Webserver started at http://127.29.156.193:40481/ using document root <none> and password file <none>
I20260807 18:21:33.530082 4506 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:33.530228 4506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:33.530632 4506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:33.533959 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "15dbbdaa2be34ff2bc600325d0ce8253"
format_stamp: "Formatted at 2026-08-07 18:21:33 on dist-test-slave-sb2z"
server_key: "31524fc6ffab98bacd3b0b985a43f954"
server_key_iv: "350a52c3e4faaeecd5f1031c63469e85"
server_key_version: "encryptionkey@0"
I20260807 18:21:33.534880 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "15dbbdaa2be34ff2bc600325d0ce8253"
format_stamp: "Formatted at 2026-08-07 18:21:33 on dist-test-slave-sb2z"
server_key: "31524fc6ffab98bacd3b0b985a43f954"
server_key_iv: "350a52c3e4faaeecd5f1031c63469e85"
server_key_version: "encryptionkey@0"
I20260807 18:21:33.540803 4506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260807 18:21:33.544901 4528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:33.546295 4506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:33.546492 4506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "15dbbdaa2be34ff2bc600325d0ce8253"
format_stamp: "Formatted at 2026-08-07 18:21:33 on dist-test-slave-sb2z"
server_key: "31524fc6ffab98bacd3b0b985a43f954"
server_key_iv: "350a52c3e4faaeecd5f1031c63469e85"
server_key_version: "encryptionkey@0"
I20260807 18:21:33.546733 4506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:33.552803 4506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:33.553669 4506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:33.553943 4506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:33.554909 4506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:33.556445 4506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:33.556577 4506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:33.556702 4506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:33.556790 4506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:33.592329 4506 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:35815
I20260807 18:21:33.592398 4669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:35815 every 8 connection(s)
I20260807 18:21:33.594303 4506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:33.600250 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 4506
I20260807 18:21:33.600443 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:33.601310 30323 external_mini_cluster.cc:1428] Setting key 1b7865ecd581b290e71121b27069d37e
I20260807 18:21:33.605654 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:33.597522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:54019 (local address 127.29.156.254:35073)
0807 18:21:33.598146 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:33.598153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:33.601530 (+ 3377us) server_negotiation.cc:407] Connection header received
0807 18:21:33.602812 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:33.602816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:33.602871 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:33.603053 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:33.604775 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:33.604781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:33.604787 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:33.604810 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:33.604864 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:33.604880 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:33.605013 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:33.605263 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 18:21:33.605328 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":328,"threads_started":1}
I20260807 18:21:33.607162 4676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:33.598287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35073 (local address 127.29.156.193:54019)
0807 18:21:33.599625 (+ 1338us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:33.599689 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:33.602519 (+ 2830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:33.603344 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:33.603356 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:33.603480 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:33.604340 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:33.604343 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:33.604399 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:33.604558 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:33.605004 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:33.605017 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:33.605026 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:21:33.605368 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 18:21:33.605681 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":361,"threads_started":1}
I20260807 18:21:33.607945 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--builtin_ntp_servers=127.29.156.212:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:21:33.609053 4671 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35073
I20260807 18:21:33.609404 4671 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:33.610172 4671 heartbeater.cc:511] Master 127.29.156.254:35073 requested a full tablet report, sending...
I20260807 18:21:33.612943 4420 ts_manager.cc:194] Registered new tserver with Master: 15dbbdaa2be34ff2bc600325d0ce8253 (127.29.156.193:35815)
W20260807 18:21:33.967435 4679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:33.967799 4679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:33.967900 4679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:33.975090 4679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:33.975193 4679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:33.975306 4679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:33.984730 4679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:33.986110 4679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:33.988128 4679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:34.002343 4687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:34.002461 4686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:34.002461 4693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:34.003365 4679 server_base.cc:1034] running on GCE node
I20260807 18:21:34.004081 4679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:34.005191 4679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:34.006453 4679 hybrid_clock.cc:648] HybridClock initialized: now 1786126894006412 us; error 67 us; skew 500 ppm
I20260807 18:21:34.006836 4679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:34.009572 4679 webserver.cc:460] Webserver started at http://127.29.156.194:35145/ using document root <none> and password file <none>
I20260807 18:21:34.010367 4679 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:34.010514 4679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:34.010874 4679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:34.013991 4679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "8d7869e08f664fe5a0845c6096aea2b1"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "75f76520002e33627892c46480839dda"
server_key_iv: "76596925ac1b89d659aa0b35d2a05843"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.014832 4679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "8d7869e08f664fe5a0845c6096aea2b1"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "75f76520002e33627892c46480839dda"
server_key_iv: "76596925ac1b89d659aa0b35d2a05843"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.020778 4679 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:21:34.024758 4700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:34.026140 4679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:34.026331 4679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "8d7869e08f664fe5a0845c6096aea2b1"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "75f76520002e33627892c46480839dda"
server_key_iv: "76596925ac1b89d659aa0b35d2a05843"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.026583 4679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:34.032744 4679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:34.033578 4679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:34.033867 4679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:34.034849 4679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:34.036406 4679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:34.036536 4679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:34.036679 4679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:34.036774 4679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:34.073216 4679 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:44345
I20260807 18:21:34.073271 4864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:44345 every 8 connection(s)
I20260807 18:21:34.074621 4679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:34.081911 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 4679
I20260807 18:21:34.082108 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:34.082674 30323 external_mini_cluster.cc:1428] Setting key 5fdd4f0a2a04194852b8ee4eaaa9b7f0
I20260807 18:21:34.083508 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.077394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:54853 (local address 127.29.156.254:35073)
0807 18:21:34.077583 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.077588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.079092 (+ 1504us) server_negotiation.cc:407] Connection header received
0807 18:21:34.080905 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.080911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.080995 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.081117 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.082642 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.082646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.082649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.082666 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.082699 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.082710 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.082848 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.083117 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.083193 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:21:34.084568 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.077983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35073 (local address 127.29.156.194:54853)
0807 18:21:34.078917 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.078963 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.080612 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.081356 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.081363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.081453 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.082295 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.082298 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.082335 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.082422 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.082894 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.082904 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.082909 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.083201 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.083455 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":288,"threads_started":1}
I20260807 18:21:34.085697 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--builtin_ntp_servers=127.29.156.212:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:21:34.086097 4868 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35073
I20260807 18:21:34.086407 4868 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:34.087002 4868 heartbeater.cc:511] Master 127.29.156.254:35073 requested a full tablet report, sending...
I20260807 18:21:34.088485 4420 ts_manager.cc:194] Registered new tserver with Master: 8d7869e08f664fe5a0845c6096aea2b1 (127.29.156.194:44345)
W20260807 18:21:34.428329 4871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:34.428627 4871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:34.428793 4871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:34.436295 4871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:34.436405 4871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:34.436597 4871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:34.446054 4871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:34.447427 4871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:34.449354 4871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:34.463474 4877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:34.463567 4879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:34.464049 4882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:34.464656 4871 server_base.cc:1034] running on GCE node
I20260807 18:21:34.465294 4871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:34.466328 4871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:34.467644 4871 hybrid_clock.cc:648] HybridClock initialized: now 1786126894467580 us; error 48 us; skew 500 ppm
I20260807 18:21:34.468005 4871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:34.470592 4871 webserver.cc:460] Webserver started at http://127.29.156.195:33299/ using document root <none> and password file <none>
I20260807 18:21:34.471400 4871 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:34.471587 4871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:34.471971 4871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:34.475108 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "c304d51820524859bb9ee9927c9fbc3d"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "60926b2f669dbe7a68308314dbd39603"
server_key_iv: "30fa582bf269c9bf9a401c1cdbdb2f22"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.476032 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "c304d51820524859bb9ee9927c9fbc3d"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "60926b2f669dbe7a68308314dbd39603"
server_key_iv: "30fa582bf269c9bf9a401c1cdbdb2f22"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.482209 4871 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260807 18:21:34.486212 4891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:34.487852 4871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:21:34.488091 4871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "c304d51820524859bb9ee9927c9fbc3d"
format_stamp: "Formatted at 2026-08-07 18:21:34 on dist-test-slave-sb2z"
server_key: "60926b2f669dbe7a68308314dbd39603"
server_key_iv: "30fa582bf269c9bf9a401c1cdbdb2f22"
server_key_version: "encryptionkey@0"
I20260807 18:21:34.488353 4871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:34.495398 4871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:34.496421 4871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:34.496764 4871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:34.497862 4871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:34.499603 4871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:34.499754 4871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:21:34.499913 4871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:34.500016 4871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:34.541745 4871 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:40289
I20260807 18:21:34.541846 5038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:40289 every 8 connection(s)
I20260807 18:21:34.543248 4871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:34.552886 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 4871
I20260807 18:21:34.552939 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.546240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:40731 (local address 127.29.156.254:35073)
0807 18:21:34.546478 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.546484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.548133 (+ 1649us) server_negotiation.cc:407] Connection header received
0807 18:21:34.550163 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.550167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.550231 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.550406 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.551999 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.552004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.552008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.552028 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.552061 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.552074 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.552183 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.552498 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.552562 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:21:34.553059 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:34.553655 30323 external_mini_cluster.cc:1428] Setting key 4ab841054cb79450421aa93ef1f9bc29
I20260807 18:21:34.554111 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.546824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35073 (local address 127.29.156.195:40731)
0807 18:21:34.547942 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.547990 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.549615 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.550647 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.550656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.550785 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.551514 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.551517 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.551598 (+ 81us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.551730 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.552184 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.552193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.552199 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.552486 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.552777 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":349,"threads_started":1}
I20260807 18:21:34.555722 5039 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35073
I20260807 18:21:34.556035 5039 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:34.556644 5039 heartbeater.cc:511] Master 127.29.156.254:35073 requested a full tablet report, sending...
I20260807 18:21:34.558163 4419 ts_manager.cc:194] Registered new tserver with Master: c304d51820524859bb9ee9927c9fbc3d (127.29.156.195:40289)
I20260807 18:21:34.570011 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:21:34.577730 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.574886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56214 (local address 127.29.156.254:35073)
0807 18:21:34.575080 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.575084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.575715 (+ 631us) server_negotiation.cc:407] Connection header received
0807 18:21:34.575897 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.575901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.575958 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.576116 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.577169 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.577174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.577177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.577195 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.577229 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.577241 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.577362 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.577431 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.577488 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:21:34.582360 4419 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:34.589727 4419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:34.617417 4671 heartbeater.cc:503] Master 127.29.156.254:35073 was elected leader, sending a full tablet report...
I20260807 18:21:34.621923 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.604853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:40289 (local address 127.0.0.1:51900)
0807 18:21:34.617948 (+ 13095us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.617970 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.618086 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.618783 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.618792 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.618899 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.619838 (+ 939us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.619842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.619881 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.619985 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.620807 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.620824 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.620838 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.621074 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.621195 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12941,"thread_start_us":13194,"threads_started":1}
I20260807 18:21:34.622644 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.604157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.29.156.194:44345)
0807 18:21:34.616538 (+ 12381us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.616555 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.617645 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 18:21:34.617966 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.617978 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.618267 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.618422 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.620929 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.620942 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.620963 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.620991 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.621074 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.621108 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.621278 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.622035 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.622167 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12109,"thread_start_us":12416,"threads_started":1}
I20260807 18:21:34.623193 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.605096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.29.156.195:40289)
0807 18:21:34.617441 (+ 12345us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.617453 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.618301 (+ 848us) server_negotiation.cc:407] Connection header received
0807 18:21:34.618380 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.618389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.618582 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.618764 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.620208 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.620222 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.620244 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.620289 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.620391 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.620422 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.622546 (+ 2124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.622631 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.622738 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12147,"thread_start_us":13106,"threads_started":1}
I20260807 18:21:34.623391 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.604071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:44345 (local address 127.0.0.1:57234)
0807 18:21:34.617439 (+ 13368us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.617480 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.617779 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.619660 (+ 1881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.619670 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.619708 (+ 38us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.620561 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.620567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.620592 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.620660 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.621354 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.621363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.621372 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.621598 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.621675 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13105,"thread_start_us":12925,"threads_started":1}
I20260807 18:21:34.627151 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.620043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33594)
0807 18:21:34.623689 (+ 3646us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.623714 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.623822 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.624870 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.624878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.624929 (+ 51us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.625534 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.625538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.625555 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.625605 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.626687 (+ 1082us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.626693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.626699 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.626826 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.626887 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3509,"thread_start_us":4251,"threads_started":1}
I20260807 18:21:34.627322 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.618537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.29.156.193:35815)
0807 18:21:34.624289 (+ 5752us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.624301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.624335 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:21:34.624418 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.624429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.624654 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.625223 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.626415 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.626426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.626439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.626456 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.626525 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.626546 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.626740 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.626819 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.626920 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5564,"thread_start_us":5374,"threads_started":1}
I20260807 18:21:34.628571 4770 tablet_service.cc:1467] Processing CreateTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 (DEFAULT_TABLE table=test-table [id=bf7741fd586645a3949c38b195cc2c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:34.628480 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 (DEFAULT_TABLE table=test-table [id=bf7741fd586645a3949c38b195cc2c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:34.630201 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68dd1e1c82654d7895b7b48aaac06291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:34.630236 4770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68dd1e1c82654d7895b7b48aaac06291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:34.631629 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 (DEFAULT_TABLE table=test-table [id=bf7741fd586645a3949c38b195cc2c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:34.633091 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68dd1e1c82654d7895b7b48aaac06291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:34.643213 5068 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Bootstrap starting.
I20260807 18:21:34.647058 5069 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Bootstrap starting.
I20260807 18:21:34.647557 5068 tablet_bootstrap.cc:654] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:34.649075 5070 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Bootstrap starting.
I20260807 18:21:34.649098 5068 log.cc:824] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:34.651389 5069 tablet_bootstrap.cc:654] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:34.652628 5070 tablet_bootstrap.cc:654] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:34.653203 5069 log.cc:824] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:34.653862 5070 log.cc:824] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:34.657130 5068 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: No bootstrap required, opened a new log
I20260807 18:21:34.657306 5070 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: No bootstrap required, opened a new log
I20260807 18:21:34.657455 5068 ts_tablet_manager.cc:1397] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260807 18:21:34.657519 5070 ts_tablet_manager.cc:1397] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:21:34.657929 5069 tablet_bootstrap.cc:492] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: No bootstrap required, opened a new log
I20260807 18:21:34.658214 5069 ts_tablet_manager.cc:1397] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 18:21:34.665405 5070 raft_consensus.cc:348] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.665851 5070 raft_consensus.cc:374] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:34.665973 5070 raft_consensus.cc:729] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15dbbdaa2be34ff2bc600325d0ce8253, State: Initialized, Role: FOLLOWER
I20260807 18:21:34.666695 5070 consensus_queue.cc:260] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.668819 5070 ts_tablet_manager.cc:1428] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260807 18:21:34.669852 5069 raft_consensus.cc:348] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.669852 5068 raft_consensus.cc:348] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.670346 5069 raft_consensus.cc:374] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:34.670337 5068 raft_consensus.cc:374] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:34.670446 5069 raft_consensus.cc:729] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7869e08f664fe5a0845c6096aea2b1, State: Initialized, Role: FOLLOWER
I20260807 18:21:34.670452 5068 raft_consensus.cc:729] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c304d51820524859bb9ee9927c9fbc3d, State: Initialized, Role: FOLLOWER
I20260807 18:21:34.671289 5069 consensus_queue.cc:260] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.671289 5068 consensus_queue.cc:260] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.672742 5039 heartbeater.cc:503] Master 127.29.156.254:35073 was elected leader, sending a full tablet report...
I20260807 18:21:34.673676 5068 ts_tablet_manager.cc:1428] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260807 18:21:34.674785 4868 heartbeater.cc:503] Master 127.29.156.254:35073 was elected leader, sending a full tablet report...
I20260807 18:21:34.675654 5069 ts_tablet_manager.cc:1428] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Time spent starting tablet: real 0.017s user 0.005s sys 0.012s
I20260807 18:21:34.785054 5079 raft_consensus.cc:482] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:34.785467 5079 raft_consensus.cc:504] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.787561 5079 leader_election.cc:288] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15dbbdaa2be34ff2bc600325d0ce8253 (127.29.156.193:35815), c304d51820524859bb9ee9927c9fbc3d (127.29.156.195:40289)
I20260807 18:21:34.791994 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.788056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:46831 (local address 127.29.156.193:35815)
0807 18:21:34.788378 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.788387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.789411 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 18:21:34.789583 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.789588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.789656 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.789765 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.791075 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.791082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.791088 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.791113 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.791159 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.791175 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.791282 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.791484 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.791592 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260807 18:21:34.792026 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.788240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:51081 (local address 127.29.156.195:40289)
0807 18:21:34.788476 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.788485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.789580 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 18:21:34.789767 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.789776 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.789842 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.789962 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.791197 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.791201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.791205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.791245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.791291 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.791309 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.791456 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.791653 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.791720 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:21:34.792128 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.788181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.29.156.194:46831)
0807 18:21:34.789239 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.789266 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.789429 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.789820 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.789825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.789851 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.790740 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.790746 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.790770 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.790831 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.791287 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.791293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.791299 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.791484 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.791630 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":880,"threads_started":1}
I20260807 18:21:34.792759 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.788189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:40289 (local address 127.29.156.194:51081)
0807 18:21:34.789426 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:34.789478 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:34.789614 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:34.790009 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:34.790012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:34.790034 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:34.790959 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:21:34.790963 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:34.790980 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:21:34.791030 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:34.791472 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:34.791478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:34.791483 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:21:34.791703 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 18:21:34.791762 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"mutex_wait_us":48,"thread_start_us":495,"threads_started":1}
I20260807 18:21:34.793303 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68dd1e1c82654d7895b7b48aaac06291" candidate_uuid: "8d7869e08f664fe5a0845c6096aea2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" is_pre_election: true
I20260807 18:21:34.794065 4612 raft_consensus.cc:2454] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7869e08f664fe5a0845c6096aea2b1 in term 0.
I20260807 18:21:34.793982 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68dd1e1c82654d7895b7b48aaac06291" candidate_uuid: "8d7869e08f664fe5a0845c6096aea2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c304d51820524859bb9ee9927c9fbc3d" is_pre_election: true
I20260807 18:21:34.794653 4980 raft_consensus.cc:2454] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7869e08f664fe5a0845c6096aea2b1 in term 0.
I20260807 18:21:34.794936 4703 leader_election.cc:302] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dbbdaa2be34ff2bc600325d0ce8253, 8d7869e08f664fe5a0845c6096aea2b1; no voters:
W20260807 18:21:34.795382 5041 tablet.cc:2369] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:34.795419 5079 raft_consensus.cc:2781] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:34.795658 5079 raft_consensus.cc:482] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:34.795792 5079 raft_consensus.cc:3037] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:34.798390 5079 raft_consensus.cc:504] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.799124 5079 leader_election.cc:288] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [CANDIDATE]: Term 1 election: Requested vote from peers 15dbbdaa2be34ff2bc600325d0ce8253 (127.29.156.193:35815), c304d51820524859bb9ee9927c9fbc3d (127.29.156.195:40289)
I20260807 18:21:34.799757 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68dd1e1c82654d7895b7b48aaac06291" candidate_uuid: "8d7869e08f664fe5a0845c6096aea2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15dbbdaa2be34ff2bc600325d0ce8253"
I20260807 18:21:34.799921 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68dd1e1c82654d7895b7b48aaac06291" candidate_uuid: "8d7869e08f664fe5a0845c6096aea2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c304d51820524859bb9ee9927c9fbc3d"
I20260807 18:21:34.800016 4612 raft_consensus.cc:3037] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:34.800217 4980 raft_consensus.cc:3037] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:34.803895 4612 raft_consensus.cc:2454] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7869e08f664fe5a0845c6096aea2b1 in term 1.
I20260807 18:21:34.803895 4980 raft_consensus.cc:2454] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7869e08f664fe5a0845c6096aea2b1 in term 1.
I20260807 18:21:34.804394 4703 leader_election.cc:302] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dbbdaa2be34ff2bc600325d0ce8253, 8d7869e08f664fe5a0845c6096aea2b1; no voters:
I20260807 18:21:34.804724 5079 raft_consensus.cc:2781] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:34.805238 5079 raft_consensus.cc:686] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 1 LEADER]: Becoming Leader. State: Replica: 8d7869e08f664fe5a0845c6096aea2b1, State: Running, Role: LEADER
I20260807 18:21:34.805806 5079 consensus_queue.cc:237] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } }
I20260807 18:21:34.811229 4420 catalog_manager.cc:5495] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d7869e08f664fe5a0845c6096aea2b1 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "8d7869e08f664fe5a0845c6096aea2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d7869e08f664fe5a0845c6096aea2b1" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 44345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 } health_report { overall_health: UNKNOWN } } }
W20260807 18:21:34.826867 4869 tablet.cc:2369] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:21:34.847215 4674 tablet.cc:2369] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:34.855360 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.853355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57242 (local address 127.29.156.194:44345)
0807 18:21:34.853539 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.853544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.853566 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:21:34.853735 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.853739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.853824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.853914 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.854717 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.854721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.854725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.854739 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.854770 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.854782 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.854873 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.854945 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.854989 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:21:34.864187 4612 raft_consensus.cc:1261] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 1 FOLLOWER]: Refusing update from remote peer 8d7869e08f664fe5a0845c6096aea2b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:34.864185 4980 raft_consensus.cc:1261] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 1 FOLLOWER]: Refusing update from remote peer 8d7869e08f664fe5a0845c6096aea2b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:34.865305 5084 consensus_queue.cc:1035] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15dbbdaa2be34ff2bc600325d0ce8253" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:34.865690 5079 consensus_queue.cc:1035] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c304d51820524859bb9ee9927c9fbc3d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:34.874604 5087 mvcc.cc:205] Tried to move back new op lower bound from 7315975761352110080 to 7315975761128783872. Current Snapshot: MvccSnapshot[applied={T|T < 7315975761352110080}]
I20260807 18:21:34.883284 5089 mvcc.cc:205] Tried to move back new op lower bound from 7315975761352110080 to 7315975761128783872. Current Snapshot: MvccSnapshot[applied={T|T < 7315975761352110080}]
I20260807 18:21:34.889570 4420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:34.889909 4420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:34.893616 4420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b4aaeafb6564d27be991ec389795c74: Sending DeleteTablet for 3 replicas of tablet 68dd1e1c82654d7895b7b48aaac06291
I20260807 18:21:34.894925 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:34 UTC) from {username='slave'} at 127.0.0.1:51900
I20260807 18:21:34.894941 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:34 UTC) from {username='slave'} at 127.0.0.1:33594
I20260807 18:21:34.895783 4770 tablet_service.cc:1514] Processing DeleteTablet for tablet 68dd1e1c82654d7895b7b48aaac06291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:34 UTC) from {username='slave'} at 127.0.0.1:57234
I20260807 18:21:34.896016 5100 tablet_replica.cc:330] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: stopping tablet replica
I20260807 18:21:34.896550 5100 raft_consensus.cc:2229] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:34.897231 5100 raft_consensus.cc:2258] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:34.897397 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 4506
I20260807 18:21:34.900082 5100 ts_tablet_manager.cc:1905] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:34.907198 5104 tablet_replica.cc:330] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: stopping tablet replica
I20260807 18:21:34.907689 5104 raft_consensus.cc:2229] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:34.908385 5104 raft_consensus.cc:2258] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:34.909257 5101 tablet_replica.cc:330] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253: stopping tablet replica
I20260807 18:21:34.909296 5100 ts_tablet_manager.cc:1918] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:34.909574 5100 log.cc:1196] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/68dd1e1c82654d7895b7b48aaac06291
I20260807 18:21:34.909732 5101 raft_consensus.cc:2229] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:34.910308 5100 ts_tablet_manager.cc:1939] T 68dd1e1c82654d7895b7b48aaac06291 P c304d51820524859bb9ee9927c9fbc3d: Deleting consensus metadata
I20260807 18:21:34.910292 5101 raft_consensus.cc:2258] T 68dd1e1c82654d7895b7b48aaac06291 P 15dbbdaa2be34ff2bc600325d0ce8253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:34.911087 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:34.899125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.29.156.193:35815)
0807 18:21:34.899306 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:34.899313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:34.904229 (+ 4916us) server_negotiation.cc:407] Connection header received
0807 18:21:34.904422 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:34.904428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:34.904539 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:34.904650 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:34.910223 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:34.910230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:34.910235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:34.910258 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:34.910305 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:34.910323 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:34.910439 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:34.910702 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:21:34.910770 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:21:34.912227 4663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:34.912498 5104 ts_tablet_manager.cc:1905] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:34.912823 4397 catalog_manager.cc:4841] TS c304d51820524859bb9ee9927c9fbc3d (127.29.156.195:40289): tablet 68dd1e1c82654d7895b7b48aaac06291 (table test-table [id=bf7741fd586645a3949c38b195cc2c1c]) successfully deleted
I20260807 18:21:34.919554 5104 ts_tablet_manager.cc:1918] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:34.919759 5104 log.cc:1196] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/68dd1e1c82654d7895b7b48aaac06291
I20260807 18:21:34.920432 5104 ts_tablet_manager.cc:1939] T 68dd1e1c82654d7895b7b48aaac06291 P 8d7869e08f664fe5a0845c6096aea2b1: Deleting consensus metadata
I20260807 18:21:34.922435 4401 catalog_manager.cc:4841] TS 8d7869e08f664fe5a0845c6096aea2b1 (127.29.156.194:44345): tablet 68dd1e1c82654d7895b7b48aaac06291 (table test-table [id=bf7741fd586645a3949c38b195cc2c1c]) successfully deleted
I20260807 18:21:35.011498 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 4506
I20260807 18:21:35.036092 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 4679
W20260807 18:21:35.036028 4401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.156.193:35815 (error 108)
I20260807 18:21:35.038983 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.038481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33610)
0807 18:21:35.038677 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.038802 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260807 18:21:35.039302 4401 catalog_manager.cc:4584] TS 15dbbdaa2be34ff2bc600325d0ce8253 (127.29.156.193:35815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68dd1e1c82654d7895b7b48aaac06291: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
I20260807 18:21:35.039889 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.037057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.29.156.194:44345)
0807 18:21:35.037263 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:35.037269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:35.037665 (+ 396us) server_negotiation.cc:407] Connection header received
0807 18:21:35.037778 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:35.037785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:35.037859 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:35.037963 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:35.039057 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:35.039064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:35.039068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:35.039089 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:35.039129 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:35.039147 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:35.039287 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:35.039510 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 18:21:35.039605 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:21:35.040614 4859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:35.093534 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.092951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33616)
0807 18:21:35.093192 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.093316 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:21:35.094765 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.094356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33630)
0807 18:21:35.094543 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.094620 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:21:35.155694 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.154994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33638)
0807 18:21:35.155239 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.155440 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 18:21:35.157352 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 4679
I20260807 18:21:35.160509 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.157233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33648)
0807 18:21:35.157380 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.157469 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 18:21:35.182173 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 4871
I20260807 18:21:35.185109 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.183076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.29.156.195:40289)
0807 18:21:35.183292 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:35.183296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:35.183316 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:21:35.183372 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:35.183376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:35.183450 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:35.183622 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:35.184425 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:35.184430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:35.184433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:35.184447 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:35.184481 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:35.184494 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:35.184600 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:35.184677 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 18:21:35.184724 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:21:35.185771 5034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:35.240895 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.240399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33664)
0807 18:21:35.240617 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.240722 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:21:35.242125 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.241688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33676)
0807 18:21:35.241886 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.241974 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:21:35.276793 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 4871
I20260807 18:21:35.300751 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 4374
I20260807 18:21:35.302067 4461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:35.416569 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.416004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35815 (local address 127.0.0.1:33680)
0807 18:21:35.416317 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:35.416419 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:35815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 18:21:35.416571 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 4374
2026-08-07T18:21:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2965 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:21:35.467229 30323 mini_oidc.cc:152] Starting JWKS server
I20260807 18:21:35.470021 30323 webserver.cc:460] Webserver started at https://127.0.0.1:34805/ using document root <none> and password file <none>
I20260807 18:21:35.470397 30323 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:21:35.471094 30323 webserver.cc:460] Webserver started at http://127.0.0.1:40827/ using document root <none> and password file <none>
2026-08-07T18:21:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:35Z Disabled control of system clock
I20260807 18:21:35.510243 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43807
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:33249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43807
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34805/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/testchainca.cert with env {}
W20260807 18:21:35.851300 5125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:35.851734 5125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:35.851902 5125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:35.859838 5125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:35.859998 5125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:35.860092 5125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:35.860215 5125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:35.860303 5125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:21:35.860399 5125 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:21:35.860464 5125 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34805/jwks.json
I20260807 18:21:35.871796 5125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34805/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:35.873425 5125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:35.874884 5125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:35.887455 5131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:35.887413 5132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:35.887483 5137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:35.888372 5125 server_base.cc:1034] running on GCE node
I20260807 18:21:35.889276 5125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:35.890812 5125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:35.892022 5125 hybrid_clock.cc:648] HybridClock initialized: now 1786126895891979 us; error 42 us; skew 500 ppm
I20260807 18:21:35.892416 5125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:35.899215 5125 webserver.cc:460] Webserver started at http://127.29.156.254:34707/ using document root <none> and password file <none>
I20260807 18:21:35.900091 5125 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:35.900195 5125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:35.900573 5125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:35.903772 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "78a0e28951a14c77aa6768cd3dddfa3d"
format_stamp: "Formatted at 2026-08-07 18:21:35 on dist-test-slave-sb2z"
server_key: "b30b838c14718e0f9d1ddf9f7e13ffe4"
server_key_iv: "8d98dfa6c522d511c4b00fb622086be1"
server_key_version: "encryptionkey@0"
I20260807 18:21:35.904652 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "78a0e28951a14c77aa6768cd3dddfa3d"
format_stamp: "Formatted at 2026-08-07 18:21:35 on dist-test-slave-sb2z"
server_key: "b30b838c14718e0f9d1ddf9f7e13ffe4"
server_key_iv: "8d98dfa6c522d511c4b00fb622086be1"
server_key_version: "encryptionkey@0"
I20260807 18:21:35.910146 5125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:21:35.913908 5145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:35.915398 5125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:35.915622 5125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "78a0e28951a14c77aa6768cd3dddfa3d"
format_stamp: "Formatted at 2026-08-07 18:21:35 on dist-test-slave-sb2z"
server_key: "b30b838c14718e0f9d1ddf9f7e13ffe4"
server_key_iv: "8d98dfa6c522d511c4b00fb622086be1"
server_key_version: "encryptionkey@0"
I20260807 18:21:35.915858 5125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:35.951645 5125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:35.952623 5125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:35.952984 5125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:35.973573 5125 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43807
I20260807 18:21:35.973672 5225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43807 every 8 connection(s)
I20260807 18:21:35.974989 5125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:35.976696 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5125
I20260807 18:21:35.976939 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:35.977757 30323 external_mini_cluster.cc:1428] Setting key 9921a9a63e5ba425b737f5b55439d5ce
I20260807 18:21:35.980070 5226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:35.994191 5226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d: Bootstrap starting.
I20260807 18:21:35.996171 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:35.980141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.29.156.254:43807)
0807 18:21:35.984382 (+ 4241us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:35.984401 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:35.984449 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:21:35.986113 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:35.986137 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:35.986621 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:35.986982 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:35.987692 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:35.989020 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:35.991065 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:35.992116 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:35.992269 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:35.993597 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:35.993604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:35.993618 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:35.994171 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:35.994265 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:35.994287 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:35.994432 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:35.994697 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:21:35.995074 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3767,"thread_start_us":3478,"threads_started":1}
I20260807 18:21:35.997684 5226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:35.998852 5226 log.cc:824] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:36.000303 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:21:36.002066 5226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d: No bootstrap required, opened a new log
I20260807 18:21:36.010416 5226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } }
I20260807 18:21:36.010833 5226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:36.010963 5226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78a0e28951a14c77aa6768cd3dddfa3d, State: Initialized, Role: FOLLOWER
I20260807 18:21:36.011605 5226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } }
I20260807 18:21:36.011873 5226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:36.012032 5226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:36.012220 5226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:36.014546 5226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } }
I20260807 18:21:36.015100 5226 leader_election.cc:302] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78a0e28951a14c77aa6768cd3dddfa3d; no voters:
I20260807 18:21:36.015930 5226 leader_election.cc:288] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:36.016049 5236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:36.017355 5236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [term 1 LEADER]: Becoming Leader. State: Replica: 78a0e28951a14c77aa6768cd3dddfa3d, State: Running, Role: LEADER
I20260807 18:21:36.017918 5236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } }
I20260807 18:21:36.018311 5226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:36.021021 5240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78a0e28951a14c77aa6768cd3dddfa3d. Latest consensus state: current_term: 1 leader_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } } }
I20260807 18:21:36.021643 5240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:36.021070 5238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a0e28951a14c77aa6768cd3dddfa3d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43807 } } }
I20260807 18:21:36.022311 5238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:36.024832 5247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:36.031227 5247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:36.040694 5247 catalog_manager.cc:1347] Generated new cluster ID: 1e59d5ae34784b06bf623455fcdf732a
I20260807 18:21:36.040839 5247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:36.053436 5247 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:36.055300 5247 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:36.065374 5247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78a0e28951a14c77aa6768cd3dddfa3d: Generated new TSK 0
I20260807 18:21:36.066397 5247 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:21:36.106176 5270 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:21:36.106420 5270 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:21:36.108870 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:36.084980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.29.156.254:43807)
0807 18:21:36.085205 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:36.085211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:36.085798 (+ 587us) server_negotiation.cc:407] Connection header received
0807 18:21:36.086153 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:36.086159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:36.086250 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:36.086411 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:21:36.087003 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.088172 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.089698 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.090395 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.090533 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:36.091094 (+ 561us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:21:36.108102 (+ 17008us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:21:36.108519 (+ 417us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 18:21:36.109022 5232 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:21:36.110814 5269 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:36.084969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43807 (local address 127.0.0.1:55910)
0807 18:21:36.085683 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:36.085700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:36.085852 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:36.086487 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:36.086491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:36.086501 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:21:36.086839 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:36.086847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.088334 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.088339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:36.089546 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:36.089558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.090611 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.090616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:36.090751 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:36.090813 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:21:36.108479 (+ 17666us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:21:36.108802 (+ 323us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.156.254:43807: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":366,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.156.254:43807: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34805/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:21:36.112795 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5125
I20260807 18:21:36.114185 5220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:36.211593 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5125
2026-08-07T18:21:36Z chronyd exiting
I20260807 18:21:36.269951 30323 test_util.cc:175] -----------------------------------------------
I20260807 18:21:36.270112 30323 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126848590636-30323-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (807 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:21:36.273916 30323 mini_oidc.cc:152] Starting JWKS server
I20260807 18:21:36.275893 30323 webserver.cc:460] Webserver started at https://127.0.0.1:36695/ using document root <none> and password file <none>
I20260807 18:21:36.276106 30323 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:21:36.276788 30323 webserver.cc:460] Webserver started at http://127.0.0.1:44923/ using document root <none> and password file <none>
2026-08-07T18:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:36Z Disabled control of system clock
I20260807 18:21:36.316407 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46189
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46189
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36695/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/testchainca.cert with env {}
W20260807 18:21:36.654314 5280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:36.654606 5280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:36.654775 5280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:36.662010 5280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:36.662133 5280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:36.662246 5280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:36.662330 5280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:36.662401 5280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:21:36.662513 5280 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:21:36.662573 5280 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36695/jwks.json
I20260807 18:21:36.672890 5280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36695/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:36.674242 5280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:36.675654 5280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:36.688359 5292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:36.688405 5295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:36.688429 5293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:36.689395 5280 server_base.cc:1034] running on GCE node
I20260807 18:21:36.690223 5280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:36.691744 5280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:36.692977 5280 hybrid_clock.cc:648] HybridClock initialized: now 1786126896692946 us; error 48 us; skew 500 ppm
I20260807 18:21:36.693346 5280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:36.695827 5280 webserver.cc:460] Webserver started at http://127.29.156.254:35663/ using document root <none> and password file <none>
I20260807 18:21:36.696516 5280 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:36.696674 5280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:36.697008 5280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:36.700037 5280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "acd6feb7a1bb47f88581591ad336b6b5"
format_stamp: "Formatted at 2026-08-07 18:21:36 on dist-test-slave-sb2z"
server_key: "632cde57c534e839c9367016c2aa5b64"
server_key_iv: "9e80ba45af3e8d5d5c1f4a5de858546e"
server_key_version: "encryptionkey@0"
I20260807 18:21:36.700909 5280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "acd6feb7a1bb47f88581591ad336b6b5"
format_stamp: "Formatted at 2026-08-07 18:21:36 on dist-test-slave-sb2z"
server_key: "632cde57c534e839c9367016c2aa5b64"
server_key_iv: "9e80ba45af3e8d5d5c1f4a5de858546e"
server_key_version: "encryptionkey@0"
I20260807 18:21:36.706199 5280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:21:36.709725 5303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:36.711176 5280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:36.711334 5280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "acd6feb7a1bb47f88581591ad336b6b5"
format_stamp: "Formatted at 2026-08-07 18:21:36 on dist-test-slave-sb2z"
server_key: "632cde57c534e839c9367016c2aa5b64"
server_key_iv: "9e80ba45af3e8d5d5c1f4a5de858546e"
server_key_version: "encryptionkey@0"
I20260807 18:21:36.711602 5280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:36.725348 5280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:36.726321 5280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:36.726608 5280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:36.747208 5280 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:46189
I20260807 18:21:36.747243 5376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:46189 every 8 connection(s)
I20260807 18:21:36.748735 5280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:36.750865 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5280
I20260807 18:21:36.751057 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:36.751796 30323 external_mini_cluster.cc:1428] Setting key 4906f47def1ec213e31c5a3ce880714e
I20260807 18:21:36.754235 5378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:36.765777 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:36.753737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.29.156.254:46189)
0807 18:21:36.754816 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:36.754828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:36.754874 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:21:36.756617 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:36.756641 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:36.757145 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:36.757602 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:36.758203 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.759657 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.760997 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.761703 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.761919 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:36.763231 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:36.763238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:36.763256 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:36.763951 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:36.764032 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:36.764055 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:36.764201 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:36.764452 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:21:36.764790 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":407,"threads_started":1}
I20260807 18:21:36.766871 5378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: Bootstrap starting.
I20260807 18:21:36.769829 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:21:36.770743 5378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:36.771947 5378 log.cc:824] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:36.775305 5378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: No bootstrap required, opened a new log
I20260807 18:21:36.783028 5378 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } }
I20260807 18:21:36.783557 5378 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:36.783717 5378 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd6feb7a1bb47f88581591ad336b6b5, State: Initialized, Role: FOLLOWER
I20260807 18:21:36.784345 5378 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } }
I20260807 18:21:36.784615 5378 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:36.784773 5378 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:36.784977 5378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:36.787273 5378 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } }
I20260807 18:21:36.787907 5378 leader_election.cc:302] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd6feb7a1bb47f88581591ad336b6b5; no voters:
I20260807 18:21:36.788605 5378 leader_election.cc:288] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:36.788844 5390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:36.790051 5390 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [term 1 LEADER]: Becoming Leader. State: Replica: acd6feb7a1bb47f88581591ad336b6b5, State: Running, Role: LEADER
I20260807 18:21:36.790853 5378 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:36.790633 5390 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } }
I20260807 18:21:36.794732 5392 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd6feb7a1bb47f88581591ad336b6b5. Latest consensus state: current_term: 1 leader_uuid: "acd6feb7a1bb47f88581591ad336b6b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } } }
I20260807 18:21:36.794762 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd6feb7a1bb47f88581591ad336b6b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd6feb7a1bb47f88581591ad336b6b5" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46189 } } }
I20260807 18:21:36.795204 5392 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:36.795212 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5 [sys.catalog]: This master's current role is: LEADER
W20260807 18:21:36.809255 5409 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:21:36.809396 5409 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:21:36.809564 5402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:36.811199 5402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:36.819666 5402 catalog_manager.cc:1347] Generated new cluster ID: 22c89dd6d2114668807624f6106921f3
I20260807 18:21:36.819785 5402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:36.835573 5402 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:36.837531 5402 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:36.852913 5402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd6feb7a1bb47f88581591ad336b6b5: Generated new TSK 0
I20260807 18:21:36.853834 5402 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:21:36.896960 5421 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:21:36.897121 5421 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:21:36.898361 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:36.877882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.29.156.254:46189)
0807 18:21:36.878101 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:36.878108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:36.878650 (+ 542us) server_negotiation.cc:407] Connection header received
0807 18:21:36.878882 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:36.878888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:36.878993 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:36.879166 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:21:36.879673 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.880853 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.882389 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.882851 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.883000 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:36.883509 (+ 509us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:21:36.897608 (+ 14099us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36695/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:21:36.898001 (+ 393us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36695/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:21:36.898520 5388 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36695/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:21:36.898590 5420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:36.877876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46189 (local address 127.0.0.1:48848)
0807 18:21:36.878538 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:36.878558 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:36.878684 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:36.879186 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:36.879189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:36.879199 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:21:36.879462 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:36.879471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.881088 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.881092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:36.882225 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:36.882238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:36.883041 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:36.883049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:36.883199 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:36.883233 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:21:36.897869 (+ 14636us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36695/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:21:36.898150 (+ 281us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.156.254:46189: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36695/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":380,"threads_started":1}
I20260807 18:21:36.899976 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5280
I20260807 18:21:36.901342 5371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:36.997996 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5280
2026-08-07T18:21:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (785 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:21:37.059648 30323 mini_oidc.cc:152] Starting JWKS server
I20260807 18:21:37.061873 30323 webserver.cc:460] Webserver started at https://127.0.0.1:41059/ using document root <none> and password file <none>
I20260807 18:21:37.062209 30323 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:21:37.063019 30323 webserver.cc:460] Webserver started at http://127.0.0.1:43839/ using document root <none> and password file <none>
2026-08-07T18:21:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:37Z Disabled control of system clock
I20260807 18:21:37.103483 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:34753
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:34753
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41059/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:21:37.455839 5430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:37.456211 5430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:37.456375 5430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:37.463722 5430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:37.463869 5430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:37.463968 5430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:37.464089 5430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:37.464177 5430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:21:37.464272 5430 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:21:37.464351 5430 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41059/jwks.json
I20260807 18:21:37.473471 5430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:34753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41059/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:37.474853 5430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:37.476794 5430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:37.489408 5436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:37.489511 5439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:37.489426 5437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:37.490361 5430 server_base.cc:1034] running on GCE node
I20260807 18:21:37.491240 5430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:37.492786 5430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:37.494065 5430 hybrid_clock.cc:648] HybridClock initialized: now 1786126897494015 us; error 58 us; skew 500 ppm
I20260807 18:21:37.494539 5430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:37.497138 5430 webserver.cc:460] Webserver started at http://127.29.156.254:37505/ using document root <none> and password file <none>
I20260807 18:21:37.497870 5430 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:37.498005 5430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:37.498345 5430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:37.501386 5430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "2f3d7048aba8477f8f2213c05753fb20"
format_stamp: "Formatted at 2026-08-07 18:21:37 on dist-test-slave-sb2z"
server_key: "a0cc4a6429fcda9937179193a1b7d956"
server_key_iv: "efccc9a5f4835832b178c2454863c6cb"
server_key_version: "encryptionkey@0"
I20260807 18:21:37.502285 5430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "2f3d7048aba8477f8f2213c05753fb20"
format_stamp: "Formatted at 2026-08-07 18:21:37 on dist-test-slave-sb2z"
server_key: "a0cc4a6429fcda9937179193a1b7d956"
server_key_iv: "efccc9a5f4835832b178c2454863c6cb"
server_key_version: "encryptionkey@0"
I20260807 18:21:37.507622 5430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:21:37.511053 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:37.512535 5430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:37.512742 5430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "2f3d7048aba8477f8f2213c05753fb20"
format_stamp: "Formatted at 2026-08-07 18:21:37 on dist-test-slave-sb2z"
server_key: "a0cc4a6429fcda9937179193a1b7d956"
server_key_iv: "efccc9a5f4835832b178c2454863c6cb"
server_key_version: "encryptionkey@0"
I20260807 18:21:37.512969 5430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:37.550096 5430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:37.551059 5430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:37.551389 5430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:37.571838 5430 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:34753
I20260807 18:21:37.571882 5522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:34753 every 8 connection(s)
I20260807 18:21:37.573238 5430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:37.577286 5523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:37.578305 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5430
I20260807 18:21:37.578547 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:37.579304 30323 external_mini_cluster.cc:1428] Setting key 8ae6604e03d6f0b31d3dbbb98b9df37c
I20260807 18:21:37.593997 5523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20: Bootstrap starting.
I20260807 18:21:37.596321 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:37.581466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.29.156.254:34753)
0807 18:21:37.582742 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:37.582760 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:37.582819 (+ 59us) server_negotiation.cc:407] Connection header received
0807 18:21:37.584932 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:37.584964 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:37.585655 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:37.586178 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:37.586751 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.588920 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.590387 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.591670 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.591854 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:37.593067 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:37.593079 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:37.593099 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:37.593905 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:37.594025 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:37.594058 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:37.594180 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:37.594456 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:21:37.594816 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":377,"threads_started":1}
I20260807 18:21:37.598413 5523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:37.599821 5523 log.cc:824] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:37.601138 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:21:37.603652 5523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20: No bootstrap required, opened a new log
I20260807 18:21:37.612237 5523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } }
I20260807 18:21:37.612730 5523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:37.612875 5523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f3d7048aba8477f8f2213c05753fb20, State: Initialized, Role: FOLLOWER
I20260807 18:21:37.613499 5523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } }
I20260807 18:21:37.613773 5523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:37.613951 5523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:37.614151 5523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:37.616852 5523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } }
I20260807 18:21:37.617484 5523 leader_election.cc:302] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f3d7048aba8477f8f2213c05753fb20; no voters:
I20260807 18:21:37.618223 5523 leader_election.cc:288] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:37.618367 5531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:37.619211 5531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [term 1 LEADER]: Becoming Leader. State: Replica: 2f3d7048aba8477f8f2213c05753fb20, State: Running, Role: LEADER
I20260807 18:21:37.619813 5531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } }
I20260807 18:21:37.620571 5523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:37.623005 5534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f3d7048aba8477f8f2213c05753fb20. Latest consensus state: current_term: 1 leader_uuid: "2f3d7048aba8477f8f2213c05753fb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } } }
I20260807 18:21:37.623481 5534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:37.624259 5532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f3d7048aba8477f8f2213c05753fb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3d7048aba8477f8f2213c05753fb20" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34753 } } }
I20260807 18:21:37.624677 5532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:37.626289 5540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:37.632342 5540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:37.642213 5540 catalog_manager.cc:1347] Generated new cluster ID: afc3abbaff474e8eb7b50afc4507700d
I20260807 18:21:37.642347 5540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:37.689851 5540 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:37.691929 5540 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:37.700169 5540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f3d7048aba8477f8f2213c05753fb20: Generated new TSK 0
I20260807 18:21:37.701052 5540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:21:37.727752 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:37.718598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.29.156.254:34753)
0807 18:21:37.718802 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:37.718809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:37.719252 (+ 443us) server_negotiation.cc:407] Connection header received
0807 18:21:37.719489 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:37.719496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:37.719629 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:37.719798 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:21:37.720336 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.721513 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.722976 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.723728 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.723881 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:37.724512 (+ 631us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:21:37.727013 (+ 2501us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:21:37.727372 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:21:37.727893 5528 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 18:21:37.729621 5567 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:37.718455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:34753 (local address 127.0.0.1:60010)
0807 18:21:37.719146 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:37.719162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:37.719289 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:37.719817 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:37.719821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:37.719830 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:21:37.720176 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:37.720188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.721659 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.721663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:37.722822 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:37.722834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:37.723992 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:37.723999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:37.724148 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:37.724189 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:21:37.727244 (+ 3055us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:21:37.729143 (+ 1899us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.156.254:34753: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":1542,"thread_start_us":400,"threads_started":1}
I20260807 18:21:37.732204 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5430
I20260807 18:21:37.733458 5518 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 18:21:37.740121 5568 webserver.cc:533] Webserver: sslize failed:
E20260807 18:21:37.740217 5568 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:21:37.832254 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5430
2026-08-07T18:21:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (837 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:21:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:37Z Disabled control of system clock
I20260807 18:21:37.936987 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44767
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44767
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/testchainca.cert with env {}
W20260807 18:21:38.282270 5573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:38.282580 5573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:38.282670 5573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:38.290030 5573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:38.290202 5573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:38.290266 5573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:38.290330 5573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:38.290376 5573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:21:38.290439 5573 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:21:38.290477 5573 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:21:38.301290 5573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:38.302670 5573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:38.304070 5573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:38.316116 5580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:38.316071 5583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:38.316150 5581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:38.317386 5573 server_base.cc:1034] running on GCE node
I20260807 18:21:38.318197 5573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:38.319643 5573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:38.320888 5573 hybrid_clock.cc:648] HybridClock initialized: now 1786126898320834 us; error 38 us; skew 500 ppm
I20260807 18:21:38.321254 5573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:21:38.323626 5573 webserver.cc:460] Webserver started at http://127.29.156.254:40681/ using document root <none> and password file <none>
I20260807 18:21:38.324329 5573 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:38.324474 5573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:38.324796 5573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:38.327685 5573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "253ee64c4c8c47c3b8897d7716080472"
format_stamp: "Formatted at 2026-08-07 18:21:38 on dist-test-slave-sb2z"
server_key: "63a549e154776f57d22bbfc5b64a77d9"
server_key_iv: "ac288fdc0127534cf2396d3b65d9d68d"
server_key_version: "encryptionkey@0"
I20260807 18:21:38.328502 5573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "253ee64c4c8c47c3b8897d7716080472"
format_stamp: "Formatted at 2026-08-07 18:21:38 on dist-test-slave-sb2z"
server_key: "63a549e154776f57d22bbfc5b64a77d9"
server_key_iv: "ac288fdc0127534cf2396d3b65d9d68d"
server_key_version: "encryptionkey@0"
I20260807 18:21:38.333740 5573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:21:38.337299 5589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:38.338773 5573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:38.339038 5573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "253ee64c4c8c47c3b8897d7716080472"
format_stamp: "Formatted at 2026-08-07 18:21:38 on dist-test-slave-sb2z"
server_key: "63a549e154776f57d22bbfc5b64a77d9"
server_key_iv: "ac288fdc0127534cf2396d3b65d9d68d"
server_key_version: "encryptionkey@0"
I20260807 18:21:38.339282 5573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:38.353837 5573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:38.354661 5573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:38.354961 5573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:38.375762 5573 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44767
I20260807 18:21:38.375790 5667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44767 every 8 connection(s)
I20260807 18:21:38.377151 5573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:38.381059 5668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:38.382292 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5573
I20260807 18:21:38.382499 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:38.383412 30323 external_mini_cluster.cc:1428] Setting key 498f63cb7e5d457df80195ef9c605df3
I20260807 18:21:38.394555 5668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472: Bootstrap starting.
I20260807 18:21:38.397181 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:38.385537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.29.156.254:44767)
0807 18:21:38.386626 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:38.386638 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:38.386680 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:21:38.388352 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:38.388380 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:38.388878 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:38.389300 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:38.389883 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.391139 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.392499 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.393183 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.393383 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:38.394784 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:38.394796 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:38.394816 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:21:38.395367 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:38.395457 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:21:38.395478 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:38.395649 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:38.395894 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 18:21:38.396247 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":389,"threads_started":1}
I20260807 18:21:38.398227 5668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:38.399330 5668 log.cc:824] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:38.401180 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:21:38.402384 5668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472: No bootstrap required, opened a new log
I20260807 18:21:38.410061 5668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } }
I20260807 18:21:38.410439 5668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:38.410560 5668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253ee64c4c8c47c3b8897d7716080472, State: Initialized, Role: FOLLOWER
I20260807 18:21:38.411170 5668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } }
I20260807 18:21:38.411383 5668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:38.411612 5668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:38.411792 5668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:38.414250 5668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } }
I20260807 18:21:38.414777 5668 leader_election.cc:302] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 253ee64c4c8c47c3b8897d7716080472; no voters:
I20260807 18:21:38.415427 5668 leader_election.cc:288] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:38.415637 5676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:38.416417 5676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [term 1 LEADER]: Becoming Leader. State: Replica: 253ee64c4c8c47c3b8897d7716080472, State: Running, Role: LEADER
I20260807 18:21:38.416914 5676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } }
I20260807 18:21:38.417557 5668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:38.419858 5677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "253ee64c4c8c47c3b8897d7716080472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } } }
I20260807 18:21:38.419900 5678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 253ee64c4c8c47c3b8897d7716080472. Latest consensus state: current_term: 1 leader_uuid: "253ee64c4c8c47c3b8897d7716080472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ee64c4c8c47c3b8897d7716080472" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44767 } } }
I20260807 18:21:38.420471 5678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:38.420780 5677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:38.422657 5683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:38.427580 5683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:38.437660 5683 catalog_manager.cc:1347] Generated new cluster ID: e5d46bea67354d8d81d2e92b842d2968
I20260807 18:21:38.437803 5683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:38.450479 5683 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:38.451869 5683 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:38.467484 5683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 253ee64c4c8c47c3b8897d7716080472: Generated new TSK 0
I20260807 18:21:38.468477 5683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:21:38.999300 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:38.488840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.29.156.254:44767)
0807 18:21:38.489105 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:38.489113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:38.489490 (+ 377us) server_negotiation.cc:407] Connection header received
0807 18:21:38.489744 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:38.489749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:38.489826 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:38.489961 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:21:38.490511 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.491277 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.493406 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.493896 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.494080 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:38.494662 (+ 582us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:21:38.998701 (+504039us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:21:38.999058 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59890: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:21:38.999452 5671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59890: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 18:21:38.999696 5706 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:38.488713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44767 (local address 127.0.0.1:59890)
0807 18:21:38.489328 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:38.489349 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:38.489520 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:38.490011 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:38.490017 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:38.490040 (+ 23us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:21:38.490342 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:38.490353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.491434 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.491440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:38.493236 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:38.493251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:38.494145 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:38.494152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:38.494339 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:38.494388 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:21:38.999008 (+504620us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:21:38.999284 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.156.254:44767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":320,"threads_started":1}
I20260807 18:21:39.001006 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5573
I20260807 18:21:39.002341 5661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:39.108809 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5573
2026-08-07T18:21:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1257 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (440 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1575 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:41 dist-test-slave-sb2z krb5kdc[5735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:41 dist-test-slave-sb2z krb5kdc[5735](info): set up 2 sockets
Aug 07 18:21:41 dist-test-slave-sb2z krb5kdc[5735](info): commencing operation
krb5kdc: starting...
W20260807 18:21:42.439116 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:42 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:21:42.720504 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:45117
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:43.084599 5756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:43.084964 5756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:43.085070 5756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:43.093477 5756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:43.093617 5756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:43.093674 5756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:43.093732 5756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:43.093784 5756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:43.105513 5756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:43.107019 5756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:43.108515 5756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:43.122116 5764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:43.122222 5765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:43.122189 5767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:43.123334 5756 server_base.cc:1034] running on GCE node
I20260807 18:21:43.124321 5756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:43.125831 5756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:43.127116 5756 hybrid_clock.cc:648] HybridClock initialized: now 1786126903127066 us; error 38 us; skew 500 ppm
I20260807 18:21:43.127446 5756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:43 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126903, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:43.131126 5756 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:21:43.132925 5756 webserver.cc:460] Webserver started at http://127.29.156.254:36039/ using document root <none> and password file <none>
I20260807 18:21:43.133697 5756 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:43.133843 5756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:43.134218 5756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:43.137619 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "11a2689e199841b4a62a10f0829a6364"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "68e2f498c74f274814793dc8ff4efcb1"
server_key_iv: "2e2afc007a3c20b6bf3131093f6981d7"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.138500 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "11a2689e199841b4a62a10f0829a6364"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "68e2f498c74f274814793dc8ff4efcb1"
server_key_iv: "2e2afc007a3c20b6bf3131093f6981d7"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.144320 5756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:21:43.148097 5778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:43.149446 5756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:21:43.149681 5756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "11a2689e199841b4a62a10f0829a6364"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "68e2f498c74f274814793dc8ff4efcb1"
server_key_iv: "2e2afc007a3c20b6bf3131093f6981d7"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.149900 5756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:43.184062 5756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:43.189265 5756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:43.189625 5756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:43.210980 5756 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:45117
I20260807 18:21:43.210983 5848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:45117 every 8 connection(s)
I20260807 18:21:43.212622 5756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:43.217025 5849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:43.221518 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5756
I20260807 18:21:43.221784 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:43.222752 30323 external_mini_cluster.cc:1428] Setting key 42c8deb2ed650d623e5317e2d564d69b
I20260807 18:21:43.233237 5849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364: Bootstrap starting.
I20260807 18:21:43.237736 5849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:43.239306 5849 log.cc:824] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:21:43 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:43.243273 5849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364: No bootstrap required, opened a new log
I20260807 18:21:43.249244 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:43.225354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.29.156.254:45117)
0807 18:21:43.226241 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:43.226255 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:43.226302 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:21:43.228281 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:43.228308 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:43.228799 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:43.229234 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:43.231318 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.232986 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.235025 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.235882 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.236038 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:43.241551 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:43.241560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:43.241577 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:43.241614 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:43.244943 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:43.245709 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:43.245716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:43.245723 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:43.245896 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:43.246318 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:43.246325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:43.246332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:43.246910 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:43.247251 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:43.247731 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 18:21:43.248202 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":409,"threads_started":1}
I20260807 18:21:43.252589 5849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } }
I20260807 18:21:43.253032 5849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:43.253155 5849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a2689e199841b4a62a10f0829a6364, State: Initialized, Role: FOLLOWER
I20260807 18:21:43.253824 5849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } }
I20260807 18:21:43.254127 5849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:43.254304 5849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:43.254520 5849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:43.257287 5849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } }
I20260807 18:21:43.257838 5849 leader_election.cc:302] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a2689e199841b4a62a10f0829a6364; no voters:
I20260807 18:21:43.258502 5849 leader_election.cc:288] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:43.258639 5854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:43.259449 5854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [term 1 LEADER]: Becoming Leader. State: Replica: 11a2689e199841b4a62a10f0829a6364, State: Running, Role: LEADER
I20260807 18:21:43.259970 5854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } }
I20260807 18:21:43.261072 5849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:43.263460 5856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a2689e199841b4a62a10f0829a6364. Latest consensus state: current_term: 1 leader_uuid: "11a2689e199841b4a62a10f0829a6364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } } }
I20260807 18:21:43.263451 5855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a2689e199841b4a62a10f0829a6364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a2689e199841b4a62a10f0829a6364" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 45117 } } }
I20260807 18:21:43.263995 5856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:43.263995 5855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:43.265728 5859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:43.270394 5859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:43.279714 5859 catalog_manager.cc:1347] Generated new cluster ID: 60795c9c48854376bed025161e3e3f08
I20260807 18:21:43.279888 5859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:43.295619 5859 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:43.297015 5859 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:43.307425 5859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a2689e199841b4a62a10f0829a6364: Generated new TSK 0
I20260807 18:21:43.308353 5859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:21:43.380045 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:43.752588 5880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:43.752887 5880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:43.753050 5880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:43.760632 5880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:43.760780 5880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:43.760948 5880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:43.771783 5880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:43.773128 5880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:43.774459 5880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:43.788897 5888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:43.788959 5887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:43.788959 5890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:43.790184 5880 server_base.cc:1034] running on GCE node
I20260807 18:21:43.790918 5880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:43.792150 5880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:43.793422 5880 hybrid_clock.cc:648] HybridClock initialized: now 1786126903793361 us; error 55 us; skew 500 ppm
I20260807 18:21:43.793810 5880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:43 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126903, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:43.797374 5880 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:21:43.799101 5880 webserver.cc:460] Webserver started at http://127.29.156.193:43155/ using document root <none> and password file <none>
I20260807 18:21:43.799964 5880 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:43.800104 5880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:43.800455 5880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:43.803520 5880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "20095206344948129312cae9aba7bd42"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "dcd425c89c8a494eb7d33580cf373031"
server_key_iv: "4e672e531ce04815f2bdfd9bf18b7719"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.804414 5880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "20095206344948129312cae9aba7bd42"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "dcd425c89c8a494eb7d33580cf373031"
server_key_iv: "4e672e531ce04815f2bdfd9bf18b7719"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.810415 5880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 18:21:43.814271 5902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:43.815796 5880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:43.815989 5880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "20095206344948129312cae9aba7bd42"
format_stamp: "Formatted at 2026-08-07 18:21:43 on dist-test-slave-sb2z"
server_key: "dcd425c89c8a494eb7d33580cf373031"
server_key_iv: "4e672e531ce04815f2bdfd9bf18b7719"
server_key_version: "encryptionkey@0"
I20260807 18:21:43.816229 5880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:43.845715 5880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:43.850548 5880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:43.850873 5880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:43.851959 5880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:43.853487 5880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:43.853614 5880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:43.853718 5880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:43.853765 5880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:43.890290 5880 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38887
I20260807 18:21:43.890512 6049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38887 every 8 connection(s)
I20260807 18:21:43.891853 5880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:43.899973 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 5880
I20260807 18:21:43.900208 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:43.901162 30323 external_mini_cluster.cc:1428] Setting key f6fe0fe2b6a063649df91faae51d1a1b
Aug 07 18:21:43 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126903, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:43.913853 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:43.894767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:48667 (local address 127.29.156.254:45117)
0807 18:21:43.895301 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:43.895306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:43.896401 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 18:21:43.898337 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:43.898343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:43.898440 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:43.898569 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:43.900459 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.901241 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.903200 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.903692 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.903825 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:43.907704 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:43.907709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:43.907712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:43.907745 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:43.910496 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:43.911314 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:43.911324 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:43.911326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:43.911452 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:43.911987 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:43.911992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:43.912001 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:43.912293 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:43.912467 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:43.913219 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 18:21:43.913471 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":276,"threads_started":1}
I20260807 18:21:43.914885 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:43.895385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:45117 (local address 127.29.156.193:48667)
0807 18:21:43.896220 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:43.896269 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:43.898047 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:43.898812 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:43.898820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:43.899353 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:43.900264 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:43.900291 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.901415 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.901420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:43.903001 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:43.903013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:43.903934 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:43.903941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:43.904193 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:43.904924 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:43.904947 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:43.907412 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:43.910684 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:43.910694 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:43.910711 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:43.911174 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:43.911639 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:43.911644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:43.911648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:43.911847 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:43.912440 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:43.912448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:43.912815 (+ 367us) client_negotiation.cc:769] Sending connection context
0807 18:21:43.913101 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:21:43.913663 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":374,"threads_started":1}
I20260807 18:21:43.916719 6051 heartbeater.cc:347] Connected to a master server at 127.29.156.254:45117
I20260807 18:21:43.917070 6051 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:43.917825 6051 heartbeater.cc:511] Master 127.29.156.254:45117 requested a full tablet report, sending...
I20260807 18:21:43.920364 5799 ts_manager.cc:194] Registered new tserver with Master: 20095206344948129312cae9aba7bd42 (127.29.156.193:38887)
I20260807 18:21:43.922742 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:48667
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:21:43.967288 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:44.312278 6059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:44.312613 6059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:44.312779 6059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:44.320356 6059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:44.320508 6059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:44.320695 6059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:44.330861 6059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:44.332326 6059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:44.333761 6059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:44.348285 6074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:44.348272 6068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:44.348554 6071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:44.349160 6059 server_base.cc:1034] running on GCE node
I20260807 18:21:44.349918 6059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:44.351053 6059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:44.352308 6059 hybrid_clock.cc:648] HybridClock initialized: now 1786126904352267 us; error 34 us; skew 500 ppm
I20260807 18:21:44.352746 6059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:44 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:44.357010 6059 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:21:44.358763 6059 webserver.cc:460] Webserver started at http://127.29.156.194:38411/ using document root <none> and password file <none>
I20260807 18:21:44.359637 6059 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:44.359803 6059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:44.360144 6059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:44.363687 6059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "9cdcc734f3ee4ab895c7f8b2932b0394"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "e7c221527da45c229545b9317d3453a9"
server_key_iv: "6e2fd9e2502f42a6e44948495320e41f"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.364573 6059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "9cdcc734f3ee4ab895c7f8b2932b0394"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "e7c221527da45c229545b9317d3453a9"
server_key_iv: "6e2fd9e2502f42a6e44948495320e41f"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.370147 6059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:21:44.374215 6086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:44.375834 6059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:21:44.376084 6059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "9cdcc734f3ee4ab895c7f8b2932b0394"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "e7c221527da45c229545b9317d3453a9"
server_key_iv: "6e2fd9e2502f42a6e44948495320e41f"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.376333 6059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:44.406090 6059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:44.410737 6059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:44.411084 6059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:44.412137 6059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:44.413591 6059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:44.413704 6059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:44.413861 6059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:44.413952 6059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:44.449667 6059 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:45135
I20260807 18:21:44.449762 6233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:45135 every 8 connection(s)
I20260807 18:21:44.451038 6059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:44.456208 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6059
I20260807 18:21:44.456409 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:44.457013 30323 external_mini_cluster.cc:1428] Setting key cde80b78578e7608bf6f931b571e7983
Aug 07 18:21:44 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:44.475274 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:44.453853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:38673 (local address 127.29.156.254:45117)
0807 18:21:44.454407 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:44.454416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:44.455393 (+ 977us) server_negotiation.cc:407] Connection header received
0807 18:21:44.458096 (+ 2703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:44.458103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:44.458202 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:44.458330 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:44.460332 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:44.461208 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:44.463871 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:44.464369 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:44.464480 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:44.469021 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:44.469026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:44.469030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:44.469064 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:44.472001 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:44.472781 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:44.472785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:44.472788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:44.472913 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:44.473394 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:44.473398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:44.473400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:44.473675 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:44.473856 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:44.474633 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 18:21:44.474870 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":246,"threads_started":1}
I20260807 18:21:44.476542 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:44.454467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:45117 (local address 127.29.156.194:38673)
0807 18:21:44.455211 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:44.455258 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:44.457807 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:44.458604 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:44.458613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:44.459225 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:44.460138 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:44.460166 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:44.461368 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:44.461386 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:44.463688 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:44.463707 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:44.464620 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:44.464629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:44.464972 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:44.465811 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:44.465846 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:44.468762 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:44.472158 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:44.472167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:44.472185 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:44.472636 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:44.473043 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:44.473048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:44.473052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:44.473260 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:44.473829 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:44.473838 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:44.474253 (+ 415us) client_negotiation.cc:769] Sending connection context
0807 18:21:44.474543 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 18:21:44.475115 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":303,"threads_started":1}
I20260807 18:21:44.478137 6235 heartbeater.cc:347] Connected to a master server at 127.29.156.254:45117
I20260807 18:21:44.478479 6235 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:44.479233 6235 heartbeater.cc:511] Master 127.29.156.254:45117 requested a full tablet report, sending...
I20260807 18:21:44.480865 5799 ts_manager.cc:194] Registered new tserver with Master: 9cdcc734f3ee4ab895c7f8b2932b0394 (127.29.156.194:45135)
I20260807 18:21:44.481863 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:38673
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:21:44.522872 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:44.875597 6242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:44.875918 6242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:44.876032 6242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:44.883934 6242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:44.884069 6242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:44.884187 6242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:44.894452 6242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:44.895813 6242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:44.897578 6242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:44.911943 6247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:44.911983 6248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:44.912024 6250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:44.912840 6242 server_base.cc:1034] running on GCE node
I20260807 18:21:44.913512 6242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:44.914531 6242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:44.915733 6242 hybrid_clock.cc:648] HybridClock initialized: now 1786126904915662 us; error 64 us; skew 500 ppm
I20260807 18:21:44.916083 6242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:44 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:44.919621 6242 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:21:44.921492 6242 webserver.cc:460] Webserver started at http://127.29.156.195:35025/ using document root <none> and password file <none>
I20260807 18:21:44.922224 6242 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:44.922363 6242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:44.922703 6242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:44.925974 6051 heartbeater.cc:503] Master 127.29.156.254:45117 was elected leader, sending a full tablet report...
I20260807 18:21:44.925904 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "9fd61d1ee63e46ef803b34680001de74"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "b819d36ffc9d6f94748a6ded8cfdd0c1"
server_key_iv: "2e6214f1d59a87b82ecd039f23fe2fee"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.926841 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "9fd61d1ee63e46ef803b34680001de74"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "b819d36ffc9d6f94748a6ded8cfdd0c1"
server_key_iv: "2e6214f1d59a87b82ecd039f23fe2fee"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.932394 6242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:21:44.936015 6262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:44.937427 6242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:21:44.937630 6242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "9fd61d1ee63e46ef803b34680001de74"
format_stamp: "Formatted at 2026-08-07 18:21:44 on dist-test-slave-sb2z"
server_key: "b819d36ffc9d6f94748a6ded8cfdd0c1"
server_key_iv: "2e6214f1d59a87b82ecd039f23fe2fee"
server_key_version: "encryptionkey@0"
I20260807 18:21:44.937860 6242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:44.967751 6242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:44.972430 6242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:44.972744 6242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:44.973767 6242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:44.975206 6242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:44.975318 6242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:44.975519 6242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:44.975641 6242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:21:45.012552 6242 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45995
I20260807 18:21:45.012650 6419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45995 every 8 connection(s)
I20260807 18:21:45.013911 6242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:45.022239 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6242
I20260807 18:21:45.022454 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:45.022894 30323 external_mini_cluster.cc:1428] Setting key 9233f945d6b745be5ea047c7a6d7faeb
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:21:45.036103 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.016852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:34491 (local address 127.29.156.254:45117)
0807 18:21:45.017353 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.017360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.018774 (+ 1414us) server_negotiation.cc:407] Connection header received
0807 18:21:45.020985 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.020991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.021082 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.021190 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.022938 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.023965 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.026267 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.026672 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.026769 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.030371 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.030377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.030380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.030415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.033138 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.033889 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.033893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.033895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.034035 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.034428 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.034431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.034433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.034731 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.034858 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.035625 (+ 767us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.035811 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":227,"threads_started":1}
I20260807 18:21:45.037349 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.017511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:45117 (local address 127.29.156.195:34491)
0807 18:21:45.018581 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:45.018628 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:45.020629 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:45.021465 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:45.021473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:45.022047 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:45.022738 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:45.022759 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.024157 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.024162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:45.026068 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:45.026085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.026880 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.026886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:45.027101 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.027815 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:45.027837 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:45.030157 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:45.033286 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:45.033294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:45.033308 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:45.033771 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:45.034137 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:45.034140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:45.034142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:45.034332 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:45.034861 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:45.034867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:45.035221 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 18:21:45.035465 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:21:45.035975 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":590,"threads_started":1}
I20260807 18:21:45.039172 6420 heartbeater.cc:347] Connected to a master server at 127.29.156.254:45117
I20260807 18:21:45.039566 6420 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:45.040274 6420 heartbeater.cc:511] Master 127.29.156.254:45117 requested a full tablet report, sending...
I20260807 18:21:45.041790 5799 ts_manager.cc:194] Registered new tserver with Master: 9fd61d1ee63e46ef803b34680001de74 (127.29.156.195:45995)
I20260807 18:21:45.042735 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:34491
I20260807 18:21:45.051966 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/test-admin.keytab.
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.086684 30323 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:21:45.092905 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:45.090041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.29.156.193:38887)
0807 18:21:45.090794 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.090801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.090829 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:21:45.090889 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.090896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.091131 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.091315 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.092424 (+ 1109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50594: BlockingRecv error: recv got EOF from 127.0.0.1:50594 (error 108)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":419,"threads_started":1}
W20260807 18:21:45.093299 30323 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc)
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.095080 30323 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.096680 30323 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:45.113850 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.098483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.29.156.193:38887)
0807 18:21:45.098741 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.098749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.098954 (+ 205us) server_negotiation.cc:407] Connection header received
0807 18:21:45.099212 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.099216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.099281 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.099373 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.100467 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.101524 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.102872 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.103456 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.103638 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.107650 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.107663 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.107677 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.107712 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.110682 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.111520 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.111573 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.111578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.111723 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.112141 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.112146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.112148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.112559 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.112809 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.113177 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.113525 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260807 18:21:45.121181 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:45.118799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.29.156.193:38887)
0807 18:21:45.119036 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.119042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.119402 (+ 360us) server_negotiation.cc:407] Connection header received
0807 18:21:45.119635 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.119639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.119711 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.119807 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.120891 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50622: BlockingRecv error: recv got EOF from 127.0.0.1:50622 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260807 18:21:45.121215 30323 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.122988 30323 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.124563 30323 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:45.141081 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.126357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.29.156.193:38887)
0807 18:21:45.126602 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.126606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.126716 (+ 110us) server_negotiation.cc:407] Connection header received
0807 18:21:45.126872 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.126877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.126941 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.127032 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.128245 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.129105 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.130366 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.130804 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.130931 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.134790 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.134796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.134800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.134840 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.138017 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.138868 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.138874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.138878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.139036 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.139467 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.139476 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.139480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.139875 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.140200 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.140492 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.140739 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260807 18:21:45.146736 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:21:45.144636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.29.156.193:38887)
0807 18:21:45.144892 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.144900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.145257 (+ 357us) server_negotiation.cc:407] Connection header received
0807 18:21:45.145459 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.145469 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.145559 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.145676 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.146439 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50650: BlockingRecv error: recv got EOF from 127.0.0.1:50650 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260807 18:21:45.147042 30323 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126848590636-30323-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.148892 30323 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:45.150368 30323 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:21:45.166355 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.151973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.29.156.193:38887)
0807 18:21:45.152332 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.152341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.152650 (+ 309us) server_negotiation.cc:407] Connection header received
0807 18:21:45.152864 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.152869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.152932 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.153085 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.154199 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.154998 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.156290 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.156765 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.156874 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.160536 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.160541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.160545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.160583 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.163658 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.164399 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.164403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.164405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.164602 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.164960 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.164965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.164968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.165257 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.165535 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.165785 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.166034 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260807 18:21:45.168387 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5880
I20260807 18:21:45.181449 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.169204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.29.156.193:38887)
0807 18:21:45.169424 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.169433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.169765 (+ 332us) server_negotiation.cc:407] Connection header received
0807 18:21:45.169896 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.169899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.169955 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.170041 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.171200 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.172142 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.173393 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.173845 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.173942 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.176127 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.176132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.176135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.176166 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.178791 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.179384 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.179388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.179390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.179597 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.180095 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.180099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.180102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.180386 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.180666 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.180865 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.181129 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:21:45.182132 6044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:45.268518 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5880
I20260807 18:21:45.293696 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6059
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:21:45.310905 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.294718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.29.156.194:45135)
0807 18:21:45.295294 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.295301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.295328 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:21:45.295389 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.295396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.295700 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.295883 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.296970 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.298011 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.299287 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.299990 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.300132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.304354 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.304368 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.304383 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.304422 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.307727 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.308495 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.308503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.308506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.308652 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.309093 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.309100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.309104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.309478 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.309820 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.310236 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.310524 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":295,"threads_started":1}
I20260807 18:21:45.312479 6225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:45.405927 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6059
I20260807 18:21:45.429224 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6242
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[5735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:21:45.447274 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:45.430325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.29.156.195:45995)
0807 18:21:45.431045 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:45.431053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:45.431087 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:21:45.431154 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:45.431161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:45.431395 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:45.431622 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:45.432879 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.434019 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.435442 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:45.436218 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:45.436384 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:45.440938 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:45.440951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:45.440966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:45.441001 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:45.444361 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.445093 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.445100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.445103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.445226 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:45.445646 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:45.445651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:45.445656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:45.446043 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:45.446391 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:45.446682 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 18:21:45.446947 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":346,"threads_started":1}
I20260807 18:21:45.448654 6413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:45.532989 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6242
I20260807 18:21:45.557909 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 5756
I20260807 18:21:45.559279 5843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:45.670826 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 5756
2026-08-07T18:21:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4549 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[6457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[6457](info): set up 2 sockets
Aug 07 18:21:45 dist-test-slave-sb2z krb5kdc[6457](info): commencing operation
krb5kdc: starting...
W20260807 18:21:46.992494 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:47 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab.
Entry for principal oryx/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab.
I20260807 18:21:47.275208 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40821
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab
--principal=oryx/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:47.619629 6472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:47.619949 6472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:47.620045 6472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:47.627334 6472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:47.627444 6472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:47.627490 6472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:47.627626 6472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:47.627681 6472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:47.637740 6472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab
--principal=oryx/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:47.639118 6472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:47.640487 6472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:47.652779 6478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:47.652819 6484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:47.652767 6481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:47.653756 6472 server_base.cc:1034] running on GCE node
I20260807 18:21:47.654568 6472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:47.656148 6472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:47.657361 6472 hybrid_clock.cc:648] HybridClock initialized: now 1786126907657310 us; error 43 us; skew 500 ppm
I20260807 18:21:47.657724 6472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:47 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:47.661012 6472 init.cc:377] Logged in from keytab as oryx/127.29.156.254@KRBTEST.COM (short username oryx)
I20260807 18:21:47.662696 6472 webserver.cc:460] Webserver started at http://127.29.156.254:36853/ using document root <none> and password file <none>
I20260807 18:21:47.663375 6472 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:47.663477 6472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:47.663796 6472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:47.666775 6472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "6ce4dfbda84045698f18de062d602c01"
format_stamp: "Formatted at 2026-08-07 18:21:47 on dist-test-slave-sb2z"
server_key: "c6cc96b2296f3530465505b0a5902941"
server_key_iv: "7ed5e2e8e067fcccc8ff64a0a0686461"
server_key_version: "encryptionkey@0"
I20260807 18:21:47.667565 6472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "6ce4dfbda84045698f18de062d602c01"
format_stamp: "Formatted at 2026-08-07 18:21:47 on dist-test-slave-sb2z"
server_key: "c6cc96b2296f3530465505b0a5902941"
server_key_iv: "7ed5e2e8e067fcccc8ff64a0a0686461"
server_key_version: "encryptionkey@0"
I20260807 18:21:47.672819 6472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:21:47.676374 6496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:47.677809 6472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:47.678064 6472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "6ce4dfbda84045698f18de062d602c01"
format_stamp: "Formatted at 2026-08-07 18:21:47 on dist-test-slave-sb2z"
server_key: "c6cc96b2296f3530465505b0a5902941"
server_key_iv: "7ed5e2e8e067fcccc8ff64a0a0686461"
server_key_version: "encryptionkey@0"
I20260807 18:21:47.678287 6472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:47.694139 6472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:47.698769 6472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:47.699179 6472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:47.719470 6472 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:40821
I20260807 18:21:47.719537 6566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:40821 every 8 connection(s)
I20260807 18:21:47.721005 6472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:21:47.723181 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6472
I20260807 18:21:47.723392 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:21:47.724126 30323 external_mini_cluster.cc:1428] Setting key ece6bc9803451f1a6c7f2f9a8fba036b
I20260807 18:21:47.725143 6567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:47.738216 6567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Bootstrap starting.
Aug 07 18:21:47 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:47.741900 6567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:47.743038 6567 log.cc:824] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:47.746740 6567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: No bootstrap required, opened a new log
I20260807 18:21:47.747865 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:47.726158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.29.156.254:40821)
0807 18:21:47.726973 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:47.726985 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:47.727028 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:21:47.728741 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:47.728765 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:47.729206 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:47.729630 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:47.731215 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:47.732653 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:47.733975 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:47.734730 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:47.734953 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:47.739462 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:47.739470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:47.739485 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:47.739578 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:47.742824 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:47.743712 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:47.743727 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:47.743741 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:47.743928 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:47.744416 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:47.744420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:47.744422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:47.745184 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:47.745501 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:47.745888 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 18:21:47.746366 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":356,"threads_started":1}
I20260807 18:21:47.755343 6567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } }
I20260807 18:21:47.755806 6567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:47.755890 6567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce4dfbda84045698f18de062d602c01, State: Initialized, Role: FOLLOWER
I20260807 18:21:47.756397 6567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } }
I20260807 18:21:47.756599 6567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:21:47.756767 6567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:21:47.756949 6567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:47.759665 6567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } }
I20260807 18:21:47.760236 6567 leader_election.cc:302] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ce4dfbda84045698f18de062d602c01; no voters:
I20260807 18:21:47.760919 6567 leader_election.cc:288] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:21:47.761170 6573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:47.762068 6573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce4dfbda84045698f18de062d602c01, State: Running, Role: LEADER
I20260807 18:21:47.762698 6573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } }
I20260807 18:21:47.763202 6567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:47.766196 6574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } }
I20260807 18:21:47.766489 6575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ce4dfbda84045698f18de062d602c01. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } }
I20260807 18:21:47.767068 6575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:47.767071 6574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:47.768230 6582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:47.775329 6582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:47.785774 6582 catalog_manager.cc:1347] Generated new cluster ID: 089e4010c63644dabbfbbb1cc1e97e96
I20260807 18:21:47.785936 6582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:47.801544 6582 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:21:47.803597 6582 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:47.811767 6582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Generated new TSK 0
I20260807 18:21:47.812922 6582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
I20260807 18:21:47.888648 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:48.249831 6600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:48.250208 6600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:48.250379 6600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:48.258082 6600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:48.258250 6600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:48.258430 6600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:48.269404 6600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:48.270856 6600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:48.272508 6600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:48.287411 6607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:48.287351 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:48.287408 6608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:48.288300 6600 server_base.cc:1034] running on GCE node
I20260807 18:21:48.288915 6600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:48.289909 6600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:48.291198 6600 hybrid_clock.cc:648] HybridClock initialized: now 1786126908291125 us; error 57 us; skew 500 ppm
I20260807 18:21:48.291643 6600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:48 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126908, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:48.295267 6600 init.cc:377] Logged in from keytab as oryx/127.29.156.193@KRBTEST.COM (short username oryx)
I20260807 18:21:48.296967 6600 webserver.cc:460] Webserver started at http://127.29.156.193:45625/ using document root <none> and password file <none>
I20260807 18:21:48.297745 6600 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:48.297904 6600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:48.298231 6600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:48.301402 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "4618ad46e641436b97dcc19e5e85f398"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "b0a817d12ed6a22964ad14fd5cbf68cc"
server_key_iv: "79268f4b1a5618f272b0990b724a2a3c"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.302304 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "4618ad46e641436b97dcc19e5e85f398"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "b0a817d12ed6a22964ad14fd5cbf68cc"
server_key_iv: "79268f4b1a5618f272b0990b724a2a3c"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.308224 6600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:21:48.312405 6618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.313827 6600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:21:48.314069 6600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "4618ad46e641436b97dcc19e5e85f398"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "b0a817d12ed6a22964ad14fd5cbf68cc"
server_key_iv: "79268f4b1a5618f272b0990b724a2a3c"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.314260 6600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:48.334192 6600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:48.339191 6600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:48.339605 6600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:48.340723 6600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:48.342218 6600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:48.342294 6600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.342469 6600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:48.342535 6600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.378613 6600 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38675
I20260807 18:21:48.378666 6778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38675 every 8 connection(s)
I20260807 18:21:48.380067 6600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:48.389817 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6600
I20260807 18:21:48.390151 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:21:48.391222 30323 external_mini_cluster.cc:1428] Setting key 9a823dfb04fc88034e873ed7769542e6
Aug 07 18:21:48 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126908, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:48.404244 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:48.382965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:58089 (local address 127.29.156.254:40821)
0807 18:21:48.383491 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:48.383495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:48.384570 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 18:21:48.386585 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:48.386591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:48.386689 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:48.386820 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:48.388561 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.389309 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.391133 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.391683 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.391829 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:48.396358 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:48.396368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:48.396375 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:48.396425 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:48.400351 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:48.401380 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:48.401389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:48.401395 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:48.401581 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:48.402059 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:48.402068 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:48.402073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:48.402476 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:48.402651 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:48.403456 (+ 805us) server_negotiation.cc:299] Negotiation successful
0807 18:21:48.403765 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":309,"threads_started":1}
I20260807 18:21:48.405218 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:48.383580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:58089)
0807 18:21:48.384363 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:48.384410 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:48.386271 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:48.387092 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:48.387101 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:48.387731 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:48.388354 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:48.388374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.389499 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.389503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:48.390935 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:48.390949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.391950 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.391958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:48.392296 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:48.393225 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:48.393273 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:48.396092 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:48.400559 (+ 4467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:48.400568 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:48.400583 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:48.401181 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:48.401714 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:48.401718 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:48.401720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:48.401929 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:48.402612 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:48.402619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:48.403016 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 18:21:48.403293 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 18:21:48.403864 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":302,"threads_started":1}
I20260807 18:21:48.407195 6780 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:48.407603 6780 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:48.408584 6780 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:48.411450 6519 ts_manager.cc:194] Registered new tserver with Master: 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:48.413800 6519 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:58089
WARNING: no policy specified for oryx/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
I20260807 18:21:48.460609 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:48.818096 6790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:48.818444 6790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:48.818547 6790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:48.826308 6790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:48.826437 6790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:48.826620 6790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:48.837059 6790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:48.838560 6790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:48.840062 6790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:48.854691 6801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:48.854763 6798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:48.854770 6799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:48.855777 6790 server_base.cc:1034] running on GCE node
I20260807 18:21:48.856405 6790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:48.857528 6790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:48.858814 6790 hybrid_clock.cc:648] HybridClock initialized: now 1786126908858743 us; error 70 us; skew 500 ppm
I20260807 18:21:48.859242 6790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:48 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126908, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:48.862977 6790 init.cc:377] Logged in from keytab as oryx/127.29.156.194@KRBTEST.COM (short username oryx)
I20260807 18:21:48.864892 6790 webserver.cc:460] Webserver started at http://127.29.156.194:44995/ using document root <none> and password file <none>
I20260807 18:21:48.865648 6790 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:48.865805 6790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:48.866140 6790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:48.869283 6790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "7dd987eb3710479fb470e4567fa43e0d"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "eb3b61d026b876faaf3fb9caeda7ca5d"
server_key_iv: "dc90d9cbd0872d0e11a84150e4ae5d43"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.870102 6790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "7dd987eb3710479fb470e4567fa43e0d"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "eb3b61d026b876faaf3fb9caeda7ca5d"
server_key_iv: "dc90d9cbd0872d0e11a84150e4ae5d43"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.875649 6790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:21:48.879307 6811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.880915 6790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:48.881150 6790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "7dd987eb3710479fb470e4567fa43e0d"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "eb3b61d026b876faaf3fb9caeda7ca5d"
server_key_iv: "dc90d9cbd0872d0e11a84150e4ae5d43"
server_key_version: "encryptionkey@0"
I20260807 18:21:48.881405 6790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:48.912659 6790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:48.917429 6790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:48.917747 6790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:48.918859 6790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:48.920560 6790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:48.920665 6790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.920822 6790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:48.920928 6790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:48.958022 6790 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:34889
I20260807 18:21:48.958123 6956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:34889 every 8 connection(s)
I20260807 18:21:48.959381 6790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:48.961925 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6790
I20260807 18:21:48.962163 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:21:48.962842 30323 external_mini_cluster.cc:1428] Setting key c1114bfa0c925cd0851593e0c78de077
Aug 07 18:21:48 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126908, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:48.982657 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:48.962345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:43675 (local address 127.29.156.254:40821)
0807 18:21:48.962913 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:48.962920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:48.964107 (+ 1187us) server_negotiation.cc:407] Connection header received
0807 18:21:48.966016 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:48.966025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:48.966120 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:48.966249 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:48.967828 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.969107 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.970908 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.971694 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.971891 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:48.975814 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:48.975830 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:48.975838 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:48.975885 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:48.978776 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:48.979797 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:48.979801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:48.979804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:48.979956 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:48.980546 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:48.980550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:48.980552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:48.980864 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:48.981098 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:48.982008 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 18:21:48.982296 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":278,"threads_started":1}
I20260807 18:21:48.984277 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:48.963067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:43675)
0807 18:21:48.963952 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:48.964008 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:48.965742 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:48.966443 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:48.966451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:48.966978 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:48.967604 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:48.967632 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.969316 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.969320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:48.970736 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:48.970751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:48.972007 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:48.972018 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:48.972254 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:48.973004 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:48.973040 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:48.975573 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:48.978928 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:48.978940 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:48.978960 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:48.979651 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:48.980093 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:48.980100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:48.980107 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:48.980407 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:48.981053 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:48.981064 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:48.981554 (+ 490us) client_negotiation.cc:769] Sending connection context
0807 18:21:48.981880 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 18:21:48.982632 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":414,"threads_started":1}
I20260807 18:21:48.986039 6959 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:48.986440 6959 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:48.987284 6959 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:48.989212 6519 ts_manager.cc:194] Registered new tserver with Master: 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889)
I20260807 18:21:48.990156 6519 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:43675
WARNING: no policy specified for oryx/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
I20260807 18:21:49.031916 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:49.390300 6969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:49.390653 6969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:49.390839 6969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:49.399083 6969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:49.399240 6969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:49.399415 6969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:49.409608 6969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:49.410991 6969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:49.412451 6969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:21:49.418169 6780 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
W20260807 18:21:49.426587 6975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:49.426548 6976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:49.426628 6978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:49.427682 6969 server_base.cc:1034] running on GCE node
I20260807 18:21:49.428260 6969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:49.429220 6969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:49.430444 6969 hybrid_clock.cc:648] HybridClock initialized: now 1786126909430375 us; error 58 us; skew 500 ppm
I20260807 18:21:49.430755 6969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:49 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126909, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:49.433923 6969 init.cc:377] Logged in from keytab as oryx/127.29.156.195@KRBTEST.COM (short username oryx)
I20260807 18:21:49.435431 6969 webserver.cc:460] Webserver started at http://127.29.156.195:36807/ using document root <none> and password file <none>
I20260807 18:21:49.436162 6969 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:49.436270 6969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:49.436522 6969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:49.439383 6969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "339595331858460fb21cc29c97346d87"
format_stamp: "Formatted at 2026-08-07 18:21:49 on dist-test-slave-sb2z"
server_key: "0a143389a4abe51de0a4b29ca8cd8ddb"
server_key_iv: "8b64cc9c5b1cc1838d81fa511375c76c"
server_key_version: "encryptionkey@0"
I20260807 18:21:49.440282 6969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "339595331858460fb21cc29c97346d87"
format_stamp: "Formatted at 2026-08-07 18:21:49 on dist-test-slave-sb2z"
server_key: "0a143389a4abe51de0a4b29ca8cd8ddb"
server_key_iv: "8b64cc9c5b1cc1838d81fa511375c76c"
server_key_version: "encryptionkey@0"
I20260807 18:21:49.445513 6969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:21:49.449301 6989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:49.450942 6969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:21:49.451118 6969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "339595331858460fb21cc29c97346d87"
format_stamp: "Formatted at 2026-08-07 18:21:49 on dist-test-slave-sb2z"
server_key: "0a143389a4abe51de0a4b29ca8cd8ddb"
server_key_iv: "8b64cc9c5b1cc1838d81fa511375c76c"
server_key_version: "encryptionkey@0"
I20260807 18:21:49.451297 6969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:49.474577 6969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:49.479585 6969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:49.479935 6969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:49.481164 6969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:49.482846 6969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:49.482933 6969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:49.483126 6969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:49.483213 6969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:49.522105 6969 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:36725
I20260807 18:21:49.522148 7135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:36725 every 8 connection(s)
I20260807 18:21:49.523658 6969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:49.531303 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 6969
I20260807 18:21:49.531478 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:21:49.531960 30323 external_mini_cluster.cc:1428] Setting key 203e19a38e81cf37ca8e98b682e7a7f1
Aug 07 18:21:49 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126909, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:49.545900 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:49.526491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:42731 (local address 127.29.156.254:40821)
0807 18:21:49.527005 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:49.527011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:49.528230 (+ 1219us) server_negotiation.cc:407] Connection header received
0807 18:21:49.530248 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:49.530252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:49.530323 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:49.530423 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:49.532457 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:49.533315 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:49.534913 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:49.535326 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:49.535464 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:49.539176 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:49.539181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:49.539185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:49.539220 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:49.542234 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:49.543175 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:49.543180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:49.543183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:49.543316 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:49.543883 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:49.543893 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:49.543895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:49.544195 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:49.544377 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:49.545164 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 18:21:49.545425 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":233,"threads_started":1}
I20260807 18:21:49.547312 7142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:49.527146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:42731)
0807 18:21:49.528008 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:49.528097 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:49.529929 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:49.530722 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:49.530731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:49.531314 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:49.532250 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:49.532278 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:49.533501 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:49.533505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:49.534758 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:49.534771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:49.535593 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:49.535603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:49.535829 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:49.536490 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:49.536514 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:49.538932 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:49.542449 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:49.542459 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:49.542481 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:49.542963 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:49.543444 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:49.543450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:49.543454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:49.543744 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:49.544350 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:49.544359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:49.544740 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 18:21:49.545044 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 18:21:49.545655 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":369,"threads_started":1}
I20260807 18:21:49.548965 7137 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:49.549256 7137 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:49.550009 7137 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:49.551679 6518 ts_manager.cc:194] Registered new tserver with Master: 339595331858460fb21cc29c97346d87 (127.29.156.195:36725)
I20260807 18:21:49.552615 6518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:42731
I20260807 18:21:49.561070 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.156.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.253.keytab.
Entry for principal oryx/127.29.156.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.253.keytab.
I20260807 18:21:49.631789 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:36855
--webserver_interface=127.29.156.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.253.keytab
--principal=oryx/127.29.156.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40821,127.29.156.253:36855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:21:49.993412 6959 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
W20260807 18:21:49.998883 7146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:49.999275 7146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:49.999388 7146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:50.006897 7146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:50.007010 7146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:50.007057 7146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:50.007121 7146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:50.007167 7146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:50.017287 7146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40821,127.29.156.253:36855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.253.keytab
--principal=oryx/127.29.156.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:36855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.156.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
W20260807 18:21:50.017714 7146 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.156.254:40821,127.29.156.253:36855'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:21:50.018877 7146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:50.020417 7146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:50.032979 7153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:50.033090 7158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:50.033046 7164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:50.034248 7146 server_base.cc:1034] running on GCE node
I20260807 18:21:50.035054 7146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:50.036669 7146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:50.038017 7146 hybrid_clock.cc:648] HybridClock initialized: now 1786126910037885 us; error 43 us; skew 500 ppm
I20260807 18:21:50.038412 7146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:50.041996 7146 init.cc:377] Logged in from keytab as oryx/127.29.156.253@KRBTEST.COM (short username oryx)
I20260807 18:21:50.043776 7146 webserver.cc:460] Webserver started at http://127.29.156.253:36739/ using document root <none> and password file <none>
I20260807 18:21:50.044519 7146 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:50.044668 7146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:50.045017 7146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:50.048250 7146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/instance:
uuid: "3091390305e34940bfce4c7046aafac6"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "733aced581eaf58e0dc4b208004cbca5"
server_key_iv: "d1c52dd353f4c721585703a4a862de63"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.049096 7146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal/instance:
uuid: "3091390305e34940bfce4c7046aafac6"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "733aced581eaf58e0dc4b208004cbca5"
server_key_iv: "d1c52dd353f4c721585703a4a862de63"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.054821 7146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:21:50.058705 7179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:50.060389 7146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:50.060621 7146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "3091390305e34940bfce4c7046aafac6"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "733aced581eaf58e0dc4b208004cbca5"
server_key_iv: "d1c52dd353f4c721585703a4a862de63"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.061205 7146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:50.079416 7146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:50.084347 7146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:50.084651 7146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:50.105789 7146 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.253:36855
I20260807 18:21:50.105814 7248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.253:36855 every 8 connection(s)
I20260807 18:21:50.107291 7146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
I20260807 18:21:50.111366 7249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:50.111876 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 7146
I20260807 18:21:50.121622 7249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.253@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:50.137089 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.114288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.29.156.253:36855)
0807 18:21:50.115169 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.115181 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.115227 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:21:50.117087 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.117112 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.117540 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.117947 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.119683 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.121171 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.122478 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.123258 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.123459 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.128854 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.128863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.128874 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.128912 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.132522 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.133510 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.133519 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.133529 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.133748 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.134180 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.134185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.134189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.134755 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.135116 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.135631 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.136058 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":391,"threads_started":1}
I20260807 18:21:50.142445 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.124881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.29.156.254:40821)
0807 18:21:50.125567 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.125577 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.126115 (+ 538us) server_negotiation.cc:407] Connection header received
0807 18:21:50.126373 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.126378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.126462 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.126591 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.128274 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.129178 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.131170 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.131702 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.131866 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.135865 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.135874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.135880 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.135936 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.139369 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.140274 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.140280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.140283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.140432 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.140840 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.140844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.140850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.141197 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.141373 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.141820 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.142055 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":387,"threads_started":1}
I20260807 18:21:50.142477 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.124902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:60000)
0807 18:21:50.125946 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.125993 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.126212 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.126689 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.126698 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.127691 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.128103 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.128115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.129344 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.129351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.131002 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.131017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.131914 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.131921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.132185 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.133042 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.133092 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.135616 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.139596 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.139608 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.139616 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.140124 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.140554 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.140559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.140563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.140713 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.141334 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.141342 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.141526 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.141706 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.142056 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":663,"threads_started":1}
I20260807 18:21:50.144374 7249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.253@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
I20260807 18:21:50.157769 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.145220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42768)
0807 18:21:50.145433 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.145456 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.145570 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.145842 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.145845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.146243 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.146390 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.146398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.147368 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.147372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.148441 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.148453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.149136 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.149141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.149275 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.149866 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.149884 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.152209 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.155229 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.155237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.155241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.155700 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.156063 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.156068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.156072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.156207 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.156779 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.156784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.156933 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.157068 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.157379 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:21:50.157843 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.145186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.29.156.253:36855)
0807 18:21:50.145355 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.145360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.145526 (+ 166us) server_negotiation.cc:407] Connection header received
0807 18:21:50.145662 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.145665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.145720 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.145797 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.146515 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.147226 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.148586 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.148977 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.149080 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.152409 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.152414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.152417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.152445 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.155085 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.155813 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.155820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.155824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.155942 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.156331 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.156336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.156339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.156649 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.156805 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.157157 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.157540 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":60}
I20260807 18:21:50.163069 7249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Bootstrap starting.
I20260807 18:21:50.166574 7249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:50.167765 7249 log.cc:824] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:50.170630 7249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: No bootstrap required, opened a new log
I20260807 18:21:50.178084 7249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } }
I20260807 18:21:50.178432 7249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:50.178503 7249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3091390305e34940bfce4c7046aafac6, State: Initialized, Role: FOLLOWER
I20260807 18:21:50.179049 7249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } }
I20260807 18:21:50.179989 7256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } } }
I20260807 18:21:50.180414 7256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:50.181082 7249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:21:50.191479 7273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:21:50.191763 7273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:21:50.194262 7146 master_runner.cc:428] Detected that this master 3091390305e34940bfce4c7046aafac6 is joining an existing cluster
I20260807 18:21:50.194379 7146 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.156.253:36855
I20260807 18:21:50.223074 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.202830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.29.156.253:36855)
0807 18:21:50.203059 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.203067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.204249 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 18:21:50.204446 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.204458 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.204557 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.204662 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.206269 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.207586 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.209900 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.210797 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.210912 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.213814 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.213823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.213830 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.213887 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.218998 (+ 5111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.219899 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.219907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.219913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.220069 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.220530 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.220577 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.220581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.221031 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.221202 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.222246 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.222526 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":71,"spinlock_wait_cycles":953728}
I20260807 18:21:50.224174 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.202917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42772)
0807 18:21:50.204112 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.204140 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.204308 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.204844 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.204851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.205593 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.206067 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.206085 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.207838 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.207846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.209741 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.209760 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.211269 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.211280 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.211554 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.212291 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.212319 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.213603 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.219160 (+ 5557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.219167 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.219174 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.219778 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.220203 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.220210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.220216 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.220415 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.221281 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.221288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.221463 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.221884 (+ 421us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.223699 (+ 1815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":851,"threads_started":1}
I20260807 18:21:50.224905 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.202833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.29.156.254:40821)
0807 18:21:50.203035 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.203041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.206538 (+ 3497us) server_negotiation.cc:407] Connection header received
0807 18:21:50.206665 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.206672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.206812 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.206925 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.208625 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.209860 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.212240 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.212976 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.213113 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.216269 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.216277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.216283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.216334 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.220261 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.221167 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.221174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.221178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.221342 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.222313 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.222319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.222323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.222728 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.222880 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.223960 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.224522 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
WARNING: no policy specified for oryx/127.29.156.252@KRBTEST.COM; defaulting to no policy
I20260807 18:21:50.226342 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.203431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:60002)
0807 18:21:50.206409 (+ 2978us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.206428 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.206555 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.207141 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.207147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.208011 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.208424 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.208438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.210112 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.210118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.212035 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.212065 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.213459 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.213469 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.213699 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.214391 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.214421 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.216003 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.220441 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.220448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.220452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.220997 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.221508 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.221514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.221517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.222160 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.222978 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.222983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.223608 (+ 625us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.225392 (+ 1784us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.225739 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2836,"thread_start_us":2650,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.252@KRBTEST.COM" created.
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.254@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
I20260807 18:21:50.254058 7286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.232860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42778)
0807 18:21:50.236469 (+ 3609us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.236520 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.236751 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.237252 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.237260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.237948 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.238181 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.238190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.239193 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.239199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.240252 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.240264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.241170 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.241178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.241426 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.242207 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.242236 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.245483 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.250247 (+ 4764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.250263 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.250274 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.250944 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.251436 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.251442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.251446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.251714 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.252636 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.252646 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.252887 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.253163 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.253552 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3329,"thread_start_us":3153,"threads_started":1}
I20260807 18:21:50.255642 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.232867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.29.156.253:36855)
0807 18:21:50.233074 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.233084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.236610 (+ 3526us) server_negotiation.cc:407] Connection header received
0807 18:21:50.236970 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.236974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.237065 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.237192 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.238349 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.239058 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.240395 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.240964 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.241110 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.245715 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.245723 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.245730 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.245775 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.250071 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.251106 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.251114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.251120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.251291 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.251885 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.251891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.251895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.252380 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.252545 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.254706 (+ 2161us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.255074 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:21:50.258193 6518 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.156.253:36855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3091390305e34940bfce4c7046aafac6"
member_type: NON_VOTER
last_known_addr {
host: "127.29.156.253"
port: 36855
}
attrs {
promote: true
}
}
dest_uuid: "6ce4dfbda84045698f18de062d602c01"
cas_config_opid_index: -1
I20260807 18:21:50.259334 6518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } }
I20260807 18:21:50.263891 7213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:50.263238 7293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3091390305e34940bfce4c7046aafac6 (127.29.156.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.252.keytab.
Entry for principal oryx/127.29.156.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.252.keytab.
I20260807 18:21:50.268850 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:46805
--webserver_interface=127.29.156.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.252.keytab
--principal=oryx/127.29.156.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:21:50.272697 7213 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Refusing update from remote peer 6ce4dfbda84045698f18de062d602c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 18:21:50.273940 7293 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.156.253:36855
I20260807 18:21:50.274219 7294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 18:21:50.274780 7146 master.cc:537] Master@127.29.156.253:36855 shutting down...
I20260807 18:21:50.274387 7291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } }
I20260807 18:21:50.274962 7291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
W20260807 18:21:50.279706 6497 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 18:21:50.291009 7146 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:50.291499 7146 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:50.291788 7146 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: stopping tablet replica
W20260807 18:21:50.292588 6497 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 -> Peer 3091390305e34940bfce4c7046aafac6 (127.29.156.253:36855): Couldn't send request to peer 3091390305e34940bfce4c7046aafac6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:21:50.298596 7146 master.cc:552] Master@127.29.156.253:36855 shutdown complete.
I20260807 18:21:50.298708 7146 master_runner.cc:305] Clearing existing system tablet
I20260807 18:21:50.299095 7146 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:50.299211 7146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:50.301797 7146 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 18:21:50.303387 7311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:50.303833 7146 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:50.304164 7146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 18:21:50.304266 7146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "3091390305e34940bfce4c7046aafac6"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "733aced581eaf58e0dc4b208004cbca5"
server_key_iv: "d1c52dd353f4c721585703a4a862de63"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.307812 7146 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:50.312704 7146 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:21:50.312897 7146 log.cc:1196] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:21:50.313354 7146 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Deleting consensus metadata
I20260807 18:21:50.313632 7146 master_runner.cc:315] Copying system tablet from 127.29.156.254:40821
I20260807 18:21:50.316318 7146 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: tablet copy: Beginning tablet copy session from remote peer at address 127.29.156.254:40821
I20260807 18:21:50.331681 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.317192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.29.156.254:40821)
0807 18:21:50.317419 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.317429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.320094 (+ 2665us) server_negotiation.cc:407] Connection header received
0807 18:21:50.320343 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.320347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.320451 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.320576 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.321516 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.322313 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.323678 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.324104 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.324246 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.326104 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.326112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.326121 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.326153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.328963 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.329591 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.329597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.329600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.329733 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:50.330105 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:50.330111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:50.330115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:50.330394 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:50.330569 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.330955 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.331277 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:21:50.331707 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.317145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:60018)
0807 18:21:50.319961 (+ 2816us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.319983 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.320166 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.320603 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.320608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.321136 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.321359 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.321369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.322477 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.322482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.323463 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.323475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.324265 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.324272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.324469 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.325157 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.325175 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:50.325888 (+ 713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:50.329104 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.329110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.329114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.329463 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.329850 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:50.329855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:50.329859 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:50.329991 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:50.330522 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:50.330526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:50.330710 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.330854 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.331190 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2678,"thread_start_us":2559,"threads_started":1}
I20260807 18:21:50.332809 6545 tablet_copy_service.cc:140] P 6ce4dfbda84045698f18de062d602c01: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3091390305e34940bfce4c7046aafac6 ({username='oryx', principal='oryx/127.29.156.253@KRBTEST.COM'} at 127.0.0.1:60018)
I20260807 18:21:50.333101 6545 tablet_copy_service.cc:161] P 6ce4dfbda84045698f18de062d602c01: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3091390305e34940bfce4c7046aafac6 at {username='oryx', principal='oryx/127.29.156.253@KRBTEST.COM'} at 127.0.0.1:60018: session id = 3091390305e34940bfce4c7046aafac6-00000000000000000000000000000000
I20260807 18:21:50.336289 6545 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:21:50.338299 7146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:21:50.343057 7146 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: tablet copy: Starting download of 0 data blocks...
I20260807 18:21:50.343364 7146 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: tablet copy: Starting download of 1 WAL segments...
I20260807 18:21:50.345885 7146 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:21:50.349159 6545 tablet_copy_service.cc:342] P 6ce4dfbda84045698f18de062d602c01: Request end of tablet copy session 3091390305e34940bfce4c7046aafac6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.156.253@KRBTEST.COM'} at 127.0.0.1:60018
I20260807 18:21:50.349359 6545 tablet_copy_service.cc:433] P 6ce4dfbda84045698f18de062d602c01: ending tablet copy session 3091390305e34940bfce4c7046aafac6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3091390305e34940bfce4c7046aafac6
I20260807 18:21:50.359622 7146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:50.362293 7321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:50.368492 7322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:50.379756 7146 server_base.cc:1034] running on GCE node
W20260807 18:21:50.381000 7325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:50.381492 7146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:50.382800 7146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:50.384021 7146 hybrid_clock.cc:648] HybridClock initialized: now 1786126910383959 us; error 40 us; skew 500 ppm
I20260807 18:21:50.384284 7146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:50.386404 7146 init.cc:377] Logged in from keytab as oryx/127.29.156.253@KRBTEST.COM (short username oryx)
I20260807 18:21:50.388469 7146 webserver.cc:460] Webserver started at http://127.29.156.253:35471/ using document root <none> and password file <none>
I20260807 18:21:50.388815 7146 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:50.388913 7146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:50.391330 7146 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:21:50.393388 7337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:50.393889 7146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260807 18:21:50.394079 7146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "3091390305e34940bfce4c7046aafac6"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "733aced581eaf58e0dc4b208004cbca5"
server_key_iv: "d1c52dd353f4c721585703a4a862de63"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.394249 7146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:50.416709 7146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:50.420635 7146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:50.452102 7146 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.253:36855
I20260807 18:21:50.452114 7414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.253:36855 every 8 connection(s)
I20260807 18:21:50.453163 7146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
I20260807 18:21:50.460377 7415 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:21:50.466136 7415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Bootstrap starting.
I20260807 18:21:50.480244 7415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:21:50.480743 7415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: Bootstrap complete.
I20260807 18:21:50.481681 7415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } }
I20260807 18:21:50.482041 7415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3091390305e34940bfce4c7046aafac6, State: Initialized, Role: LEARNER
I20260807 18:21:50.482378 7415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } }
I20260807 18:21:50.483299 7418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } }
I20260807 18:21:50.483718 7415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:50.483747 7418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:21:50.491441 7433 catalog_manager.cc:1259] Loaded cluster ID: 089e4010c63644dabbfbbb1cc1e97e96
I20260807 18:21:50.491647 7433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: loading cluster ID for follower catalog manager: success
I20260807 18:21:50.494005 7433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: acquiring CA information for follower catalog manager: success
I20260807 18:21:50.496258 7433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:21:50.555369 7137 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
I20260807 18:21:50.798285 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.775073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42782)
0807 18:21:50.789879 (+ 14806us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.789912 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.790536 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.791814 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.791822 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.791854 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:50.792106 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.792118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.793874 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.793880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.795479 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.795494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.797063 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.797134 (+ 71us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.797399 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.797467 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 18:21:50.797642 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:21:50.797850 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14594,"thread_start_us":15145,"threads_started":1}
I20260807 18:21:50.798408 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.775272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.29.156.253:36855)
0807 18:21:50.791359 (+ 16087us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.791364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.791383 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:21:50.791447 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.791450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.791606 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.791771 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:50.792266 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.793715 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.795721 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.796819 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.797227 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.797454 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:50.797679 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 18:21:50.797873 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15968,"thread_start_us":15716,"threads_started":1}
I20260807 18:21:50.800185 7381 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 18:21:50.801060 7418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ce4dfbda84045698f18de062d602c01. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } }
I20260807 18:21:50.801545 7418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: LEARNER
W20260807 18:21:50.829917 7305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:50.830313 7305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:50.830521 7305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:50.839401 7305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:50.839653 7305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:50.839838 7305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:50.840031 7305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:50.840116 7305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:50.852685 7305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.252.keytab
--principal=oryx/127.29.156.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:46805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.156.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:50.854561 7305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:50.856375 7305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:50.871423 7441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:50.871513 7442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:50.871702 7444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:50.872774 7305 server_base.cc:1034] running on GCE node
I20260807 18:21:50.873795 7305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:50.875437 7305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:50.876784 7305 hybrid_clock.cc:648] HybridClock initialized: now 1786126910876700 us; error 47 us; skew 500 ppm
I20260807 18:21:50.877290 7305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:50.881772 7305 init.cc:377] Logged in from keytab as oryx/127.29.156.252@KRBTEST.COM (short username oryx)
I20260807 18:21:50.884093 7305 webserver.cc:460] Webserver started at http://127.29.156.252:39813/ using document root <none> and password file <none>
I20260807 18:21:50.884997 7305 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:50.885169 7305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:50.885752 7305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:21:50.890074 7305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/instance:
uuid: "fd7d27bf313545e8bea040a5dabe65ef"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "34369413165358fa189df2ef14e85879"
server_key_iv: "a645a419de89ea6d3bd89959b8022525"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.891016 7305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal/instance:
uuid: "fd7d27bf313545e8bea040a5dabe65ef"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "34369413165358fa189df2ef14e85879"
server_key_iv: "a645a419de89ea6d3bd89959b8022525"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.898273 7305 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.008s
I20260807 18:21:50.902940 7457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:50.904743 7305 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:21:50.905025 7305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "fd7d27bf313545e8bea040a5dabe65ef"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "34369413165358fa189df2ef14e85879"
server_key_iv: "a645a419de89ea6d3bd89959b8022525"
server_key_version: "encryptionkey@0"
I20260807 18:21:50.905280 7305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:50.938745 7305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:50.944130 7305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:50.944495 7305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:50.970464 7305 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.252:46805
I20260807 18:21:50.970537 7527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.252:46805 every 8 connection(s)
I20260807 18:21:50.972086 7305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
I20260807 18:21:50.974169 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 7305
I20260807 18:21:50.976856 7528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:50.986086 7528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
Aug 07 18:21:50 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.252@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:21:51.006014 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.976489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47162 (local address 127.29.156.252:46805)
0807 18:21:50.977628 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.977642 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.977689 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:21:50.979733 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.979759 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.980205 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.980692 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.982864 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.984635 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.986708 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.987840 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.987998 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.994403 (+ 6405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:50.994412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:50.994424 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:50.994475 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:50.999826 (+ 5351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.000974 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.000988 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.001000 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.001301 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.001844 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.001850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.001853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.002810 (+ 957us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.003061 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.004183 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.004702 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":664,"threads_started":1}
I20260807 18:21:51.010818 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.989569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.29.156.254:40821)
0807 18:21:50.990239 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:50.990247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:50.991058 (+ 811us) server_negotiation.cc:407] Connection header received
0807 18:21:50.991269 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:50.991276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:50.991370 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:50.991520 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:50.993274 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.994520 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.996907 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.997802 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.997936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.002912 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.002924 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.002928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.002980 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.007024 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.008040 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.008063 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.008069 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.008258 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.008726 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.008731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.008735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.009252 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.009464 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.010267 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.010475 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":282,"threads_started":1}
I20260807 18:21:51.010919 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:50.989254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:60034)
0807 18:21:50.990892 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:50.990936 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:50.991123 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:50.991662 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:50.991667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:50.992704 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:50.993100 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.993112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.994706 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.994716 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.996694 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:50.996740 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:50.998022 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:50.998032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:50.998309 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:50.998988 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:50.999010 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.002618 (+ 3608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.007177 (+ 4559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.007188 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.007194 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.007876 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.008424 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.008428 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.008431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.008584 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.009571 (+ 987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.009581 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.009864 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.010146 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.010420 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"thread_start_us":1206,"threads_started":1}
I20260807 18:21:51.012791 7528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.252@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
I20260807 18:21:51.027881 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.013597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42796)
0807 18:21:51.013821 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.013844 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.013942 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.014255 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.014258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.014665 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.014877 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.014885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.016152 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.016156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.017309 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.017323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.018262 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.018268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.018443 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.019107 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.019127 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.021755 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.025260 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.025265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.025268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.025798 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.026205 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.026209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.026211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.026336 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.026938 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.026942 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.027129 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.027254 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.027636 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:21:51.027978 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.013699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.29.156.253:36855)
0807 18:21:51.013883 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.013888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.013906 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:21:51.014052 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.014056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.014126 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.014216 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.015021 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.015905 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.017501 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.018067 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.018185 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.021948 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.021953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.021957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.021994 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.025097 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.025916 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.025921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.025923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.026057 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.026438 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.026442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.026444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.026793 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.026922 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.027346 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.027677 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:21:51.029140 7528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126910, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.252@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
I20260807 18:21:51.044669 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.030087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.0.0.1:47166)
0807 18:21:51.030274 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.030296 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.030410 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.030767 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.030770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.031292 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.031451 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.031461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.032634 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.032638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.033607 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.033620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.034454 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.034459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.034659 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.035333 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.035357 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.037955 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.041935 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.041939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.041942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.042403 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.042800 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.042804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.042806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.043009 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.043772 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.043776 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.043915 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.044035 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.044276 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:21:51.045104 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.030061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.29.156.252:46805)
0807 18:21:51.030255 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.030260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.030442 (+ 182us) server_negotiation.cc:407] Connection header received
0807 18:21:51.030507 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.030511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.030570 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.030656 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.031646 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.032497 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.033761 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.034302 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.034428 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.038183 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.038189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.038193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.038227 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.041788 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.042538 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.042543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.042546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.042687 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.043206 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.043210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.043212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.043656 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.043783 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.044206 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.044455 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:21:51.050472 7528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Bootstrap starting.
I20260807 18:21:51.054298 7528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:51.055575 7528 log.cc:824] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:51.058631 7528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: No bootstrap required, opened a new log
I20260807 18:21:51.067354 7528 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } }
I20260807 18:21:51.067791 7528 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:51.067927 7528 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd7d27bf313545e8bea040a5dabe65ef, State: Initialized, Role: FOLLOWER
I20260807 18:21:51.068567 7528 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } }
I20260807 18:21:51.069653 7536 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } } }
I20260807 18:21:51.070187 7536 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:51.070686 7528 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:21:51.081965 7553 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:21:51.082130 7553 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:21:51.087365 7305 master_runner.cc:428] Detected that this master fd7d27bf313545e8bea040a5dabe65ef is joining an existing cluster
I20260807 18:21:51.087596 7305 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.156.252:46805
I20260807 18:21:51.115687 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6600
I20260807 18:21:51.132047 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.092987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.29.156.253:36855)
0807 18:21:51.093210 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.093229 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.104322 (+ 11093us) server_negotiation.cc:407] Connection header received
0807 18:21:51.105634 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.105641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.105730 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.105804 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.108811 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.110253 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.113082 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.113994 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.114117 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.122906 (+ 8789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.122915 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.122921 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.122978 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.128072 (+ 5094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.129140 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.129147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.129152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.129325 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.129926 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.129933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.129937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.130347 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.130514 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.131216 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.131503 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:21:51.133940 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.093051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.29.156.254:40821)
0807 18:21:51.093399 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.093406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.108352 (+ 14946us) server_negotiation.cc:407] Connection header received
0807 18:21:51.108603 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.108609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.108710 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.108812 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.111133 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.112581 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.118584 (+ 6003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.119837 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.119973 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.124532 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.124540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.124545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.124588 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.129268 (+ 4680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.130592 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.130600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.130604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.130824 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.131739 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.131745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.131748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.132148 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.132295 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.133035 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.133357 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260807 18:21:51.134047 7561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.094423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:60048)
0807 18:21:51.108197 (+ 13774us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.108218 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.108439 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.109044 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.109050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.110251 (+ 1201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.110879 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.110894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.115253 (+ 4359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.115260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.117981 (+ 2721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.117997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.120496 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.120504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.120807 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.122600 (+ 1793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.122625 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.124271 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.129804 (+ 5533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.129809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.129812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.130394 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.130985 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.130990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.130994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.131217 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.132429 (+ 1212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.132434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.132653 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.132815 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.133230 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13586,"thread_start_us":9335,"threads_started":1}
I20260807 18:21:51.136806 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.093337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.29.156.252:46805)
0807 18:21:51.093569 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.093575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.102164 (+ 8589us) server_negotiation.cc:407] Connection header received
0807 18:21:51.102379 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.102384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.102454 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.102545 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.107636 (+ 5091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.109141 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.111731 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.112573 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.112681 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.116100 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.116108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.116114 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.116169 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.122077 (+ 5908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.129412 (+ 7335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.129419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.129423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.129669 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.133938 (+ 4269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.133946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.133950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.134453 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.134603 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.135385 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.135712 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":26496}
I20260807 18:21:51.137642 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.093312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.0.0.1:47180)
0807 18:21:51.101992 (+ 8680us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.102012 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.102240 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.102702 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.102707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.104806 (+ 2099us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.105303 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.105321 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.109347 (+ 4026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.109353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.111572 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.111594 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.113168 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.113176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.113383 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.114207 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.114230 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.115832 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.122215 (+ 6383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.122221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.122227 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.123023 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.131361 (+ 8338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.131367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.131371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.131603 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.134655 (+ 3052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.134660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.134834 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.135046 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.135393 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8510,"mutex_wait_us":33,"thread_start_us":14522,"threads_started":1}
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.193@KRBTEST.COM
I20260807 18:21:51.140837 7565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.094327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:42812)
0807 18:21:51.104204 (+ 9877us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.104225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.105481 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.106026 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.106031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.107090 (+ 1059us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.108614 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.108629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.110543 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.110551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.112861 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.112877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.114544 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.114554 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.114770 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.118174 (+ 3404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.118205 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.122602 (+ 4397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.128261 (+ 5659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.128266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.128273 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.128965 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.129539 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.129545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.129549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.129739 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.130612 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.130617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.130849 (+ 232us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.139576 (+ 8727us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.140088 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9664,"mutex_wait_us":5,"thread_start_us":6964,"threads_started":1}
I20260807 18:21:51.150898 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.117500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.29.156.193:38675)
0807 18:21:51.121788 (+ 4288us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.121799 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.121834 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:21:51.121914 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.121938 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.122294 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.125410 (+ 3116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.125875 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.128031 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.130471 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.131579 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.132523 (+ 944us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.142084 (+ 9561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.142105 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.142125 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.142177 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.146832 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.147914 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.147925 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.147929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.148120 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.148597 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.148602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.148606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.148991 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.149432 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.149850 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.150213 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4041,"thread_start_us":3820,"threads_started":1}
I20260807 18:21:51.152863 6774 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.254@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
I20260807 18:21:51.163339 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.143730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.0.0.1:45144)
0807 18:21:51.144019 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.144035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.144183 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.144729 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.144734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.145309 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.145513 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.145526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.147380 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.147386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.148909 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.148927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.150252 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.150259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.150520 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.151547 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.151576 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.154753 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.159822 (+ 5069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.159829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.159834 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.160532 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.161086 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.161093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.161098 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.161354 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.162106 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.162115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.162324 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.162496 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.162870 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260807 18:21:51.163458 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.143752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.29.156.252:46805)
0807 18:21:51.143919 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.143925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.144117 (+ 192us) server_negotiation.cc:407] Connection header received
0807 18:21:51.144358 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.144364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.144555 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.144674 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.145691 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.147217 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.149238 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.150015 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.151675 (+ 1660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.155080 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.155089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.155096 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.155148 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.159624 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.160721 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.160729 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.160734 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.160923 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.161505 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.161512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.161517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.161927 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.162128 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.162669 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.162963 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:21:51.165516 6518 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.156.252:46805: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef"
member_type: NON_VOTER
last_known_addr {
host: "127.29.156.252"
port: 46805
}
attrs {
promote: true
}
}
dest_uuid: "6ce4dfbda84045698f18de062d602c01"
cas_config_opid_index: 5
I20260807 18:21:51.166882 6518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } }
I20260807 18:21:51.169041 7294 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.183811 7294 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.156.252:46805
I20260807 18:21:51.184703 7305 master.cc:537] Master@127.29.156.252:46805 shutting down...
W20260807 18:21:51.189136 6499 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 -> Peer fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252:46805): Couldn't send request to peer fd7d27bf313545e8bea040a5dabe65ef. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:21:51.198761 7381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Refusing update from remote peer 6ce4dfbda84045698f18de062d602c01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:21:51.204236 7294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 18:21:51.204681 7571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.205310 7571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:51.211503 7381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.216835 7418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: NON_VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: true } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.217196 7418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:21:51.217762 7575 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: attempting to promote NON_VOTER 3091390305e34940bfce4c7046aafac6 to VOTER
I20260807 18:21:51.218845 7575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } }
W20260807 18:21:51.220661 7465 tablet.cc:2369] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:51.223372 7305 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:51.223807 7305 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 0 FOLLOWER]: Raft consensus is shut down!
W20260807 18:21:51.223886 6499 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 -> Peer fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252:46805): Couldn't send request to peer fd7d27bf313545e8bea040a5dabe65ef. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:21:51.224000 7305 tablet_replica.cc:330] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: stopping tablet replica
I20260807 18:21:51.227892 7381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 LEARNER]: Refusing update from remote peer 6ce4dfbda84045698f18de062d602c01: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:21:51.228808 7575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 18:21:51.231827 7305 master.cc:552] Master@127.29.156.252:46805 shutdown complete.
I20260807 18:21:51.231958 7305 master_runner.cc:305] Clearing existing system tablet
I20260807 18:21:51.232374 7305 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:51.232546 7305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:51.231995 7574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3091390305e34940bfce4c7046aafac6 (127.29.156.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.237687 7294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.237977 7294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:51.238332 7294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.238636 7294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:51.239217 7381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3091390305e34940bfce4c7046aafac6 (127.29.156.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.242142 7305 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.008s
I20260807 18:21:51.244058 7418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.244318 7418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:51.247977 7582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:51.248420 7305 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:51.248867 7305 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:21:51.249022 7305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "fd7d27bf313545e8bea040a5dabe65ef"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "34369413165358fa189df2ef14e85879"
server_key_iv: "a645a419de89ea6d3bd89959b8022525"
server_key_version: "encryptionkey@0"
I20260807 18:21:51.254173 7305 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:51.262375 7305 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:21:51.262830 7305 log.cc:1196] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:21:51.263651 7305 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Deleting consensus metadata
I20260807 18:21:51.264372 7305 master_runner.cc:315] Copying system tablet from 127.29.156.254:40821
I20260807 18:21:51.267915 7305 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: tablet copy: Beginning tablet copy session from remote peer at address 127.29.156.254:40821
I20260807 18:21:51.288094 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6600
I20260807 18:21:51.302801 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.268857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.29.156.254:40821)
0807 18:21:51.269139 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.269146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.280356 (+ 11210us) server_negotiation.cc:407] Connection header received
0807 18:21:51.280566 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.280573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.280782 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.280936 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.282382 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.283886 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.286669 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.287292 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.287435 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.293895 (+ 6460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.293906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.293911 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.293962 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.298921 (+ 4959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.300038 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.300044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.300049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.300321 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.300901 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.300908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.300912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.301316 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.301526 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.302073 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.302342 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260807 18:21:51.302947 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.268903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:38752)
0807 18:21:51.280167 (+ 11264us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.280193 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.280402 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.280982 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.280987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.281830 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.282196 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.282213 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.284052 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.284058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.286500 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.286515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.287790 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.287797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.287981 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.292455 (+ 4474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.292487 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.293624 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.299124 (+ 5500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.299132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.299139 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.299881 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.300472 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.300480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.300487 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.300739 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.301472 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.301480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.301714 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.301888 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.302296 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11035,"thread_start_us":10879,"threads_started":1}
I20260807 18:21:51.303968 6545 tablet_copy_service.cc:140] P 6ce4dfbda84045698f18de062d602c01: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fd7d27bf313545e8bea040a5dabe65ef ({username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:38752)
I20260807 18:21:51.304179 6545 tablet_copy_service.cc:161] P 6ce4dfbda84045698f18de062d602c01: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fd7d27bf313545e8bea040a5dabe65ef at {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:38752: session id = fd7d27bf313545e8bea040a5dabe65ef-00000000000000000000000000000000
I20260807 18:21:51.308277 6545 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:21:51.310520 7305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:21:51.319001 7305 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: tablet copy: Starting download of 0 data blocks...
I20260807 18:21:51.319355 7305 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: tablet copy: Starting download of 1 WAL segments...
I20260807 18:21:51.320345 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:38675
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:40821
--webserver_port=45625
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:21:51.324434 7305 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:21:51.330416 6545 tablet_copy_service.cc:342] P 6ce4dfbda84045698f18de062d602c01: Request end of tablet copy session fd7d27bf313545e8bea040a5dabe65ef-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:38752
I20260807 18:21:51.330574 6545 tablet_copy_service.cc:433] P 6ce4dfbda84045698f18de062d602c01: ending tablet copy session fd7d27bf313545e8bea040a5dabe65ef-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fd7d27bf313545e8bea040a5dabe65ef
I20260807 18:21:51.341773 7305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:51.344592 7602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:51.349808 7603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:51.358294 7305 server_base.cc:1034] running on GCE node
W20260807 18:21:51.359903 7607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:51.360469 7305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:51.361195 7305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:51.362537 7305 hybrid_clock.cc:648] HybridClock initialized: now 1786126911362486 us; error 42 us; skew 500 ppm
I20260807 18:21:51.362810 7305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126911, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:51.369318 7305 init.cc:377] Logged in from keytab as oryx/127.29.156.252@KRBTEST.COM (short username oryx)
I20260807 18:21:51.372635 7305 webserver.cc:460] Webserver started at http://127.29.156.252:43027/ using document root <none> and password file <none>
I20260807 18:21:51.372977 7305 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:51.373103 7305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:51.375792 7305 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 18:21:51.377552 7615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:51.378001 7305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 18:21:51.378180 7305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "fd7d27bf313545e8bea040a5dabe65ef"
format_stamp: "Formatted at 2026-08-07 18:21:50 on dist-test-slave-sb2z"
server_key: "34369413165358fa189df2ef14e85879"
server_key_iv: "a645a419de89ea6d3bd89959b8022525"
server_key_version: "encryptionkey@0"
I20260807 18:21:51.378369 7305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:51.400019 7305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:51.403690 7305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:51.432003 7305 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.252:46805
I20260807 18:21:51.432045 7677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.252:46805 every 8 connection(s)
I20260807 18:21:51.432968 7305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
I20260807 18:21:51.439881 7678 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:21:51.442502 7678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Bootstrap starting.
I20260807 18:21:51.458556 7678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:21:51.459117 7678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Bootstrap complete.
I20260807 18:21:51.460253 7678 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } }
I20260807 18:21:51.460561 7678 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fd7d27bf313545e8bea040a5dabe65ef, State: Initialized, Role: LEARNER
I20260807 18:21:51.460824 7678 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } }
I20260807 18:21:51.472306 7678 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:21:51.471944 7681 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.475896 7681 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: LEARNER
I20260807 18:21:51.490669 7696 catalog_manager.cc:1259] Loaded cluster ID: 089e4010c63644dabbfbbb1cc1e97e96
I20260807 18:21:51.490794 7696 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: loading cluster ID for follower catalog manager: success
I20260807 18:21:51.494632 7696 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: acquiring CA information for follower catalog manager: success
I20260807 18:21:51.497402 7696 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:21:51.622611 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.614023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.29.156.252:46805)
0807 18:21:51.614920 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.614928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.614952 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:21:51.615042 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.615047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.615153 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.615282 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:51.615889 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.617537 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.619802 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.621215 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.621435 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.621621 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.622018 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.622168 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":531,"threads_started":1}
I20260807 18:21:51.622628 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.613793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.0.0.1:45158)
0807 18:21:51.614105 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.614129 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.614280 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.615357 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.615365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.615415 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:51.615678 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.615694 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.617713 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.617722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.619620 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.619638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.621441 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.621450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.621802 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.621860 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.621972 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.622176 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260807 18:21:51.624135 7648 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 18:21:51.625226 7681 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ce4dfbda84045698f18de062d602c01. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:51.625607 7681 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: LEARNER
W20260807 18:21:51.755110 7601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:51.755412 7601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:51.755656 7601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:51.763734 7601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:51.763969 7601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:51.764191 7601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:21:51.776595 7601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:38675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=45625
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:51.778137 7601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:51.779755 7601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:51.795729 7703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:51.795781 7706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:51.796245 7711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:51.798031 7601 server_base.cc:1034] running on GCE node
I20260807 18:21:51.798861 7601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:51.799947 7601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:51.801240 7601 hybrid_clock.cc:648] HybridClock initialized: now 1786126911801167 us; error 55 us; skew 500 ppm
I20260807 18:21:51.801719 7601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126911, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:51.805974 7601 init.cc:377] Logged in from keytab as oryx/127.29.156.193@KRBTEST.COM (short username oryx)
I20260807 18:21:51.807915 7601 webserver.cc:460] Webserver started at http://127.29.156.193:45625/ using document root <none> and password file <none>
I20260807 18:21:51.808822 7601 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:51.809048 7601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:51.815758 7601 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 18:21:51.819063 7725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:51.820792 7601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:51.821034 7601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "4618ad46e641436b97dcc19e5e85f398"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "b0a817d12ed6a22964ad14fd5cbf68cc"
server_key_iv: "79268f4b1a5618f272b0990b724a2a3c"
server_key_version: "encryptionkey@0"
I20260807 18:21:51.822322 7601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:51.843418 7601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:51.848344 7601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:51.848683 7601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:51.849929 7601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:51.851640 7601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:51.851763 7601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:51.851899 7601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:51.851994 7601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:51.907186 7601 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38675
I20260807 18:21:51.907325 7881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38675 every 8 connection(s)
I20260807 18:21:51.908883 7601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:21:51.909433 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 7601
I20260807 18:21:51.909776 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6790
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126911, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126911, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126911, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
Aug 07 18:21:51 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.194@KRBTEST.COM
I20260807 18:21:51.949599 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.914936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:44827 (local address 127.29.156.252:46805)
0807 18:21:51.915168 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.915177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.921047 (+ 5870us) server_negotiation.cc:407] Connection header received
0807 18:21:51.922013 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.922023 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.922138 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.922233 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.926628 (+ 4395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.927998 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.933019 (+ 5021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.933980 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.934141 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.941001 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.941009 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.941014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.941073 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.945221 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.946386 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.946392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.946395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.946568 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.947163 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.947171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.947177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.947643 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.947793 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.948937 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.949204 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:21:51.949599 7887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.912392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40621 (local address 127.29.156.254:40821)
0807 18:21:51.925847 (+ 13455us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.925852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.925872 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:21:51.925930 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.925933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.926058 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.926191 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.927879 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.929022 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.933000 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.933894 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.934001 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.942036 (+ 8035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.942044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.942056 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.942125 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.946377 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.947225 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.947232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.947237 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.947400 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.947917 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.947921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.947924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.948275 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.948416 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.948980 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.949212 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13339,"thread_start_us":13214,"threads_started":1}
I20260807 18:21:51.951141 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.921889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:47897 (local address 127.29.156.253:36855)
0807 18:21:51.925855 (+ 3966us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.925865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.927136 (+ 1271us) server_negotiation.cc:407] Connection header received
0807 18:21:51.927224 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.927229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.927330 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.927475 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.929683 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.931155 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.934428 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.935278 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.935436 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.942320 (+ 6884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.942330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.942336 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.942392 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.946958 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.948099 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.948108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.948113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.948300 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.948899 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.948907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.948912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.949368 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.949545 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.950412 (+ 867us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.950663 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3788,"thread_start_us":3663,"threads_started":1}
I20260807 18:21:51.952527 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.914254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.29.156.193:44827)
0807 18:21:51.920866 (+ 6612us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.920910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.921652 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.922899 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.922913 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.924377 (+ 1464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.926361 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.926397 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.929566 (+ 3169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.929574 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.932763 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.932782 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.934911 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.934920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.935230 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.936363 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.936390 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.940638 (+ 4248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.945441 (+ 4803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.945452 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.945472 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.946223 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.946733 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.946739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.946748 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.947014 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.947896 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.947903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.948399 (+ 496us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.949920 (+ 1521us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.950784 (+ 864us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6429,"mutex_wait_us":463,"thread_start_us":5502,"threads_started":1}
I20260807 18:21:51.953020 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.921191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.29.156.193:47897)
0807 18:21:51.926825 (+ 5634us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.926850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.927015 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.927766 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.927775 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.928866 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.929444 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.929460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.931368 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.931376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.934208 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.934228 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.935506 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.935513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.935792 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.936847 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.936875 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.941676 (+ 4801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.947158 (+ 5482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.947167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.947173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.947899 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.948458 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.948464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.948469 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.948728 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.949626 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.949631 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.949822 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.950025 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.952628 (+ 2603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5431,"mutex_wait_us":2016,"thread_start_us":4192,"threads_started":1}
I20260807 18:21:51.953332 7889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.913246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:40621)
0807 18:21:51.915177 (+ 1931us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:51.915241 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:51.919117 (+ 3876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:51.926470 (+ 7353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:51.926478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:51.927140 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:51.927688 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.927703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.929288 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.929292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.932772 (+ 3480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:51.932787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.934442 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.934449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:51.934790 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.936235 (+ 1445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:51.936283 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:51.941743 (+ 5460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:51.946540 (+ 4797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.946544 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.946548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.947085 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.947579 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:51.947584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:51.947588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:51.947774 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:51.948503 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:51.948508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:51.948701 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 18:21:51.949920 (+ 1219us) client_negotiation.cc:240] Negotiation successful
0807 18:21:51.950669 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1429,"mutex_wait_us":140,"thread_start_us":1807,"threads_started":1}
I20260807 18:21:51.954573 7884 heartbeater.cc:347] Connected to a master server at 127.29.156.253:36855
I20260807 18:21:51.954713 7883 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:51.955098 7883 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:51.955099 7884 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:51.955228 7882 heartbeater.cc:347] Connected to a master server at 127.29.156.252:46805
I20260807 18:21:51.955368 7882 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:51.956540 7882 heartbeater.cc:511] Master 127.29.156.252:46805 requested a full tablet report, sending...
I20260807 18:21:51.957048 7883 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:51.957558 7884 heartbeater.cc:511] Master 127.29.156.253:36855 requested a full tablet report, sending...
I20260807 18:21:51.959304 6518 ts_manager.cc:194] Re-registered known tserver with Master: 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:51.960007 7635 ts_manager.cc:194] Registered new tserver with Master: 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:51.960018 7365 ts_manager.cc:194] Registered new tserver with Master: 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:51.962209 6518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:40621
I20260807 18:21:51.962621 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:51.911371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.29.156.194:34889)
0807 18:21:51.918683 (+ 7312us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:51.918693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:51.929225 (+ 10532us) server_negotiation.cc:407] Connection header received
0807 18:21:51.929321 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:51.929331 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:51.929718 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:51.929899 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:51.932814 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.934780 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.937022 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:51.938207 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:51.938374 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:51.946505 (+ 8131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:51.946530 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:51.946552 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:51.946607 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:51.951650 (+ 5043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.954295 (+ 2645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.954307 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.954311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.954527 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:51.955302 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:51.955308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:51.955311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:51.955889 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:51.956362 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:51.956644 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:21:51.956974 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7117,"thread_start_us":6932,"threads_started":1}
I20260807 18:21:51.964533 6952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:52.052278 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6790
I20260807 18:21:52.076721 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:34889
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:40821
--webserver_port=44995
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:21:52.191211 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: NON_VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: true } } }
I20260807 18:21:52.191828 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:52.193567 7902 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: attempting to promote NON_VOTER fd7d27bf313545e8bea040a5dabe65ef to VOTER
I20260807 18:21:52.194413 7902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:52.196831 7648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 LEARNER]: Refusing update from remote peer 6ce4dfbda84045698f18de062d602c01: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:21:52.197345 7381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Refusing update from remote peer 6ce4dfbda84045698f18de062d602c01: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:21:52.197788 7903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:21:52.198304 7901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:21:52.200831 7901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.201633 7381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.210891 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.210805 7902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.211236 7902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:52.211567 7902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.211855 7902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:52.211241 7906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.211979 7906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:52.212203 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:52.211354 7647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.216691 7907 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ce4dfbda84045698f18de062d602c01" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:52.217171 7907 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:21:52.445860 7900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:52.446179 7900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:52.446354 7900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:52.453877 7900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:52.454044 7900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:52.454219 7900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:21:52.465148 7900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:34889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=44995
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:52.466552 7900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:52.468254 7900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:52.483019 7914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:52.483062 7913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:52.483232 7917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:52.484400 7900 server_base.cc:1034] running on GCE node
I20260807 18:21:52.485235 7900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:52.486337 7900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:52.487627 7900 hybrid_clock.cc:648] HybridClock initialized: now 1786126912487554 us; error 68 us; skew 500 ppm
I20260807 18:21:52.487954 7900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:52 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126912, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:52.491257 7900 init.cc:377] Logged in from keytab as oryx/127.29.156.194@KRBTEST.COM (short username oryx)
I20260807 18:21:52.492769 7900 webserver.cc:460] Webserver started at http://127.29.156.194:44995/ using document root <none> and password file <none>
I20260807 18:21:52.493451 7900 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:52.493603 7900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:52.499490 7900 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:21:52.502451 7931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:52.504069 7900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:52.504212 7900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "7dd987eb3710479fb470e4567fa43e0d"
format_stamp: "Formatted at 2026-08-07 18:21:48 on dist-test-slave-sb2z"
server_key: "eb3b61d026b876faaf3fb9caeda7ca5d"
server_key_iv: "dc90d9cbd0872d0e11a84150e4ae5d43"
server_key_version: "encryptionkey@0"
I20260807 18:21:52.505311 7900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:52.544013 7900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:52.548803 7900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:52.549137 7900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:52.550241 7900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:52.551733 7900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:52.551833 7900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:52.551981 7900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:52.552098 7900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:52.589648 7900 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:34889
I20260807 18:21:52.589706 8086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:34889 every 8 connection(s)
I20260807 18:21:52.591148 7900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:21:52.597222 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 7900
I20260807 18:21:52.597633 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6969
Aug 07 18:21:52 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126912, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
Aug 07 18:21:52 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126912, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
Aug 07 18:21:52 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.195@KRBTEST.COM
Aug 07 18:21:52 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126912, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
I20260807 18:21:52.663519 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.594438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:53045 (local address 127.29.156.254:40821)
0807 18:21:52.638999 (+ 44561us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:52.639031 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:52.639059 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:21:52.639137 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:52.639146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:52.639253 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:52.639408 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:52.641921 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.642766 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.646223 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.647012 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.647137 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.653833 (+ 6696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:52.653838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:52.653842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:52.653887 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:52.658671 (+ 4784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.660019 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.660023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.660026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.660158 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.660688 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.660693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.660696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.661163 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:52.661323 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:52.662820 (+ 1497us) server_negotiation.cc:299] Negotiation successful
0807 18:21:52.663059 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44396,"thread_start_us":44260,"threads_started":1}
I20260807 18:21:52.663936 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.597800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:58501 (local address 127.29.156.253:36855)
0807 18:21:52.632611 (+ 34811us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:52.632618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:52.634101 (+ 1483us) server_negotiation.cc:407] Connection header received
0807 18:21:52.634209 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:52.634218 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:52.634323 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:52.634433 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:52.643009 (+ 8576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.644247 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.646174 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.646848 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.647024 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.653651 (+ 6627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:52.653660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:52.653665 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:52.653712 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:52.659355 (+ 5643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.660583 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.660591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.660597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.660838 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.661498 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.661504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.661508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.661886 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:52.662089 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:52.662974 (+ 885us) server_negotiation.cc:299] Negotiation successful
0807 18:21:52.663331 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34631,"thread_start_us":34811,"threads_started":1}
I20260807 18:21:52.665557 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.597160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.29.156.194:58501)
0807 18:21:52.633750 (+ 36590us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:52.633778 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:52.633966 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:52.635316 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:52.635334 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:52.636882 (+ 1548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:52.638051 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.638086 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.644452 (+ 6366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.644467 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.646026 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.646037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.648443 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.648448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.648569 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.649405 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:52.649427 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:52.653325 (+ 3898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:52.659583 (+ 6258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.659589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.659593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.660405 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.661050 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.661055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.661059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.661337 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.662167 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:52.662171 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:52.662362 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 18:21:52.662602 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 18:21:52.664385 (+ 1783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36428,"mutex_wait_us":1282,"thread_start_us":35572,"threads_started":1}
I20260807 18:21:52.665759 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.595395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:53045)
0807 18:21:52.596604 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:52.604788 (+ 8184us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:52.613163 (+ 8375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:52.639498 (+ 26335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:52.639508 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:52.640375 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:52.640958 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.640978 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.642954 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.642970 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.645541 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.645557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.647544 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.647557 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.647895 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.649091 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:52.649116 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:52.653626 (+ 4510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:52.658872 (+ 5246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.658883 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.658909 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.659813 (+ 904us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.660280 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.660286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.660289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.660563 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.661615 (+ 1052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:52.661623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:52.662150 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 18:21:52.662604 (+ 454us) client_negotiation.cc:240] Negotiation successful
0807 18:21:52.663404 (+ 800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":4,"spinlock_wait_cycles":25344,"thread_start_us":7411,"threads_started":1}
I20260807 18:21:52.667987 8088 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:52.668431 8088 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:52.669639 8088 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:52.670869 8089 heartbeater.cc:347] Connected to a master server at 127.29.156.253:36855
I20260807 18:21:52.671020 8089 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:52.671269 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.598919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53648 (local address 127.29.156.195:36725)
0807 18:21:52.641212 (+ 42293us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:52.641220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:52.641248 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:21:52.641304 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:52.641310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:52.641531 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:52.641663 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:52.644891 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.647358 (+ 2467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.650193 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.652940 (+ 2747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.653093 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.662160 (+ 9067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:52.662178 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:52.662197 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:52.662254 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:52.667135 (+ 4881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.668332 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.668341 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.668345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.668564 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.669173 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.669178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.669182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.669621 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:52.669976 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:52.670448 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 18:21:52.670762 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42094,"thread_start_us":41885,"threads_started":1}
I20260807 18:21:52.671762 8089 heartbeater.cc:511] Master 127.29.156.253:36855 requested a full tablet report, sending...
I20260807 18:21:52.672262 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.596413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.29.156.194:60887)
0807 18:21:52.631361 (+ 34948us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:52.631395 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:52.634778 (+ 3383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:52.648177 (+ 13399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:52.648184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:52.649314 (+ 1130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:52.649844 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.649863 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.651725 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.651729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.653326 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:52.653345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.656897 (+ 3552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.656919 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:52.657153 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.657789 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:52.657813 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:52.661407 (+ 3594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:52.668863 (+ 7456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.668868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.668872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.669481 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.669978 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:52.669984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:52.669990 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:52.670167 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:52.670914 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:52.670920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:52.671179 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:21:52.671373 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:21:52.671770 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34713,"thread_start_us":41795,"threads_started":1}
I20260807 18:21:52.673079 7129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:52.673339 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:52.596359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:60887 (local address 127.29.156.252:46805)
0807 18:21:52.647789 (+ 51430us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:52.647795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:52.647816 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:21:52.647875 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:52.647880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:52.647976 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:52.648096 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:52.650106 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.651558 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.653591 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:52.656603 (+ 3012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:52.657490 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:52.661635 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:52.661642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:52.661646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:52.661689 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:52.668703 (+ 7014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.669641 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.669648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.669651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.669819 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:52.670347 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:52.670353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:52.670357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:52.670749 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:52.672521 (+ 1772us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:52.672660 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:21:52.672879 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51282,"thread_start_us":51172,"threads_started":1}
I20260807 18:21:52.673629 7365 ts_manager.cc:194] Registered new tserver with Master: 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889)
I20260807 18:21:52.674356 8087 heartbeater.cc:347] Connected to a master server at 127.29.156.252:46805
I20260807 18:21:52.674520 8087 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:52.675019 8087 heartbeater.cc:511] Master 127.29.156.252:46805 requested a full tablet report, sending...
I20260807 18:21:52.676555 7635 ts_manager.cc:194] Registered new tserver with Master: 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889)
I20260807 18:21:52.678345 6518 ts_manager.cc:194] Re-registered known tserver with Master: 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889)
I20260807 18:21:52.679392 6518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:53045
I20260807 18:21:52.765027 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6969
I20260807 18:21:52.794333 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:36725
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:40821
--webserver_port=36807
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:21:52.965348 7883 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
W20260807 18:21:53.150187 8106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:53.150486 8106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:53.150667 8106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:53.158176 8106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:53.158340 8106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:53.158489 8106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:21:53.168771 8106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:36725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=36807
--tserver_master_addrs=127.29.156.254:40821,127.29.156.253:36855,127.29.156.252:46805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:53.170279 8106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:53.171681 8106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:53.186781 8117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:53.186793 8116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:53.186992 8119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:53.187793 8106 server_base.cc:1034] running on GCE node
I20260807 18:21:53.188606 8106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:53.189759 8106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:53.191037 8106 hybrid_clock.cc:648] HybridClock initialized: now 1786126913190971 us; error 81 us; skew 500 ppm
I20260807 18:21:53.191430 8106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:53 dist-test-slave-sb2z krb5kdc[6457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126913, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:21:53.195264 8106 init.cc:377] Logged in from keytab as oryx/127.29.156.195@KRBTEST.COM (short username oryx)
I20260807 18:21:53.197032 8106 webserver.cc:460] Webserver started at http://127.29.156.195:36807/ using document root <none> and password file <none>
I20260807 18:21:53.197806 8106 fs_manager.cc:359] Metadata directory not provided
I20260807 18:21:53.197979 8106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:21:53.204246 8106 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:21:53.207248 8129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:53.208906 8106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:21:53.209086 8106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "339595331858460fb21cc29c97346d87"
format_stamp: "Formatted at 2026-08-07 18:21:49 on dist-test-slave-sb2z"
server_key: "0a143389a4abe51de0a4b29ca8cd8ddb"
server_key_iv: "8b64cc9c5b1cc1838d81fa511375c76c"
server_key_version: "encryptionkey@0"
I20260807 18:21:53.210237 8106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:21:53.242758 8106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:21:53.247157 8106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:21:53.247478 8106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:21:53.248617 8106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:21:53.250099 8106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:21:53.250190 8106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:53.250342 8106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:21:53.250449 8106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:21:53.288446 8106 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:36725
I20260807 18:21:53.288563 8291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:36725 every 8 connection(s)
I20260807 18:21:53.290000 8106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:21:53.294174 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 8106
Aug 07 18:21:53 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126913, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
Aug 07 18:21:53 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126913, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.252@KRBTEST.COM
Aug 07 18:21:53 dist-test-slave-sb2z krb5kdc[6457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126913, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.253@KRBTEST.COM
I20260807 18:21:53.345700 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.293446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33865 (local address 127.29.156.254:40821)
0807 18:21:53.306880 (+ 13434us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.306891 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.308869 (+ 1978us) server_negotiation.cc:407] Connection header received
0807 18:21:53.312027 (+ 3158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.312033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.312157 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.312293 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.314195 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.315158 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.318607 (+ 3449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.319335 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.319490 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.333987 (+ 14497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.333997 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.334004 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.334087 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.338247 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.339800 (+ 1553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.339806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.339810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.340016 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.342830 (+ 2814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.342838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.342841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.343376 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.343600 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.344833 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.345151 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13226,"thread_start_us":13449,"threads_started":1}
I20260807 18:21:53.347357 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.294065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:33865)
0807 18:21:53.308219 (+ 14154us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.308290 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.311699 (+ 3409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.312605 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.312616 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.313241 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:53.313989 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.314010 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.315334 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.315343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.318437 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.318458 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.319635 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.319644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.320174 (+ 530us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.321394 (+ 1220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:53.321446 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:53.333590 (+ 12144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:53.338404 (+ 4814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.338416 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.338439 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.339179 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.340149 (+ 970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.340154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.340159 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.340549 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.343582 (+ 3033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:53.343596 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:53.344263 (+ 667us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.344669 (+ 406us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.345371 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13388,"thread_start_us":25684,"threads_started":1}
I20260807 18:21:53.350708 8294 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40821
I20260807 18:21:53.351176 8294 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:53.352846 8294 heartbeater.cc:511] Master 127.29.156.254:40821 requested a full tablet report, sending...
I20260807 18:21:53.355247 6518 ts_manager.cc:194] Re-registered known tserver with Master: 339595331858460fb21cc29c97346d87 (127.29.156.195:36725)
I20260807 18:21:53.357154 6518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:33865
I20260807 18:21:53.362834 8301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.294075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.29.156.195:49293)
0807 18:21:53.320718 (+ 26643us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.320738 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.320994 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.322595 (+ 1601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.322601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.323316 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:53.323721 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.323736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.325508 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.325516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.329277 (+ 3761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.329303 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.330594 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.330601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.330792 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.331577 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:53.331605 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:53.335924 (+ 4319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:53.349118 (+ 13194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.349126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.349134 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.350594 (+ 1460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.353175 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.353184 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.353191 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.353489 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.359266 (+ 5777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:53.359276 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:53.359656 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.359964 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.361040 (+ 1076us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25234,"mutex_wait_us":497,"thread_start_us":24987,"threads_started":1}
I20260807 18:21:53.366251 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.294504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:49293 (local address 127.29.156.252:46805)
0807 18:21:53.310318 (+ 15814us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.310323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.320830 (+ 10507us) server_negotiation.cc:407] Connection header received
0807 18:21:53.322024 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.322031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.322131 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.322225 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.323967 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.325298 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.329469 (+ 4171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.330166 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.330301 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.336322 (+ 6021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.336334 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.336343 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.336404 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.348905 (+ 12501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.352552 (+ 3647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.352563 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.352570 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.352848 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.357908 (+ 5060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.357918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.357925 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.358756 (+ 831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.359146 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.364082 (+ 4936us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.364484 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15696,"thread_start_us":15532,"threads_started":1}
I20260807 18:21:53.367772 8292 heartbeater.cc:347] Connected to a master server at 127.29.156.252:46805
I20260807 18:21:53.367938 8292 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:53.368607 8292 heartbeater.cc:511] Master 127.29.156.252:46805 requested a full tablet report, sending...
I20260807 18:21:53.370478 7635 ts_manager.cc:194] Registered new tserver with Master: 339595331858460fb21cc29c97346d87 (127.29.156.195:36725)
I20260807 18:21:53.376863 8299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.293450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:58487 (local address 127.29.156.253:36855)
0807 18:21:53.319005 (+ 25555us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.319013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.320490 (+ 1477us) server_negotiation.cc:407] Connection header received
0807 18:21:53.320733 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.320741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.320931 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.321051 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.324178 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.325487 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.330332 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.331083 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.331247 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.336982 (+ 5735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.336991 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.336996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.337073 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.342617 (+ 5544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.370169 (+ 27552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.370180 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.370183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.370395 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.375156 (+ 4761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.375164 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.375170 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.375628 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.375793 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.376295 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.376484 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25402,"thread_start_us":26227,"threads_started":1}
I20260807 18:21:53.377092 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.342051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.29.156.252:46805)
0807 18:21:53.342844 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.342850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.346037 (+ 3187us) server_negotiation.cc:407] Connection header received
0807 18:21:53.346452 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.346461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.346577 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.346716 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.352480 (+ 5764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.354223 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.358637 (+ 4414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.359434 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.359562 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.366035 (+ 6473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.366046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.366052 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.366098 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.371353 (+ 5255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.372503 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.372510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.372515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.372750 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.373808 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.373814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.373818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.374224 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.374485 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.376129 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.376455 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":502,"threads_started":1}
I20260807 18:21:53.379791 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.294065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.29.156.195:58487)
0807 18:21:53.320326 (+ 26261us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.320349 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.320505 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.321291 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.321297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.322644 (+ 1347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:21:53.323112 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.323126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.325697 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.325708 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.330130 (+ 4422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.330145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.331606 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.331614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.331837 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.333455 (+ 1618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:21:53.333480 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:21:53.336734 (+ 3254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:21:53.369003 (+ 32269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.369013 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.369022 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.369964 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.373354 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:21:53.373363 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:21:53.373368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:21:53.373592 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:21:53.375746 (+ 2154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:21:53.375752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:21:53.375960 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.377572 (+ 1612us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.377990 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26067,"mutex_wait_us":717,"thread_start_us":12415,"threads_started":1}
I20260807 18:21:53.380955 8296 heartbeater.cc:347] Connected to a master server at 127.29.156.253:36855
I20260807 18:21:53.381199 8296 heartbeater.cc:464] Registering TS with master...
I20260807 18:21:53.381803 8296 heartbeater.cc:511] Master 127.29.156.253:36855 requested a full tablet report, sending...
I20260807 18:21:53.381827 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.343946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.29.156.254:40821)
0807 18:21:53.344695 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.344701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.344718 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:21:53.344771 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.344776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.344854 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.348186 (+ 3332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.349102 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.354402 (+ 5300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.356715 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.360930 (+ 4215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.361084 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.365408 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.365416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.365437 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.365498 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.374726 (+ 9228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.375885 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.375891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.375894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.376123 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.377584 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.377589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.377592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.377993 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.381054 (+ 3061us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.381225 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.381469 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":572,"threads_started":1}
I20260807 18:21:53.383144 7365 ts_manager.cc:194] Registered new tserver with Master: 339595331858460fb21cc29c97346d87 (127.29.156.195:36725)
I20260807 18:21:53.397687 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.344377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52084 (local address 127.29.156.253:36855)
0807 18:21:53.374370 (+ 29993us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.374380 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.375322 (+ 942us) server_negotiation.cc:407] Connection header received
0807 18:21:53.375567 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.375576 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.375668 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.375793 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.378840 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.380281 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.384573 (+ 4292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.385464 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.385620 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.389198 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.389220 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.389225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.389280 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.393617 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.394450 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.394457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.394461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.394800 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.395441 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.395447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.395450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.396013 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.396347 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.396714 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.397048 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29794,"thread_start_us":30029,"threads_started":1}
I20260807 18:21:53.387256 6518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:53.399986 6518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:53.435026 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.422906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:36725 (local address 127.0.0.1:53650)
0807 18:21:53.424643 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.424662 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.424805 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.425939 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.425944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.425976 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:53.426496 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.426513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.429440 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.429444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.431060 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.431073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.434089 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.434095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.434298 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.434335 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.434433 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.434578 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1572,"thread_start_us":1415,"threads_started":1}
I20260807 18:21:53.435243 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.423335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.29.156.195:36725)
0807 18:21:53.424177 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.424187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.424865 (+ 678us) server_negotiation.cc:407] Connection header received
0807 18:21:53.424999 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.425008 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.425413 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.425969 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:53.426743 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.429206 (+ 2463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.431283 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.433822 (+ 2539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.434072 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.434311 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.434511 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.434742 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":465,"threads_started":1}
I20260807 18:21:53.440305 8194 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7420566d66434290813037a201798f (DEFAULT_TABLE table=test-table [id=040943adeb9e4ab99ac0881134397a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:53.441213 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.426203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38675 (local address 127.0.0.1:60376)
0807 18:21:53.427982 (+ 1779us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.428024 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.428163 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.428822 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.428826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.428852 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:53.431562 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.431576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.433619 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.433624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.437269 (+ 3645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.437284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.440174 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.440182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.440376 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.440414 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.440494 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.440637 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1680,"thread_start_us":1509,"threads_started":1}
I20260807 18:21:53.442400 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.425854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.29.156.193:38675)
0807 18:21:53.427253 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.427266 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.428139 (+ 873us) server_negotiation.cc:407] Connection header received
0807 18:21:53.428324 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.428330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.428634 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.428849 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:53.431840 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.433440 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.437472 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.439062 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.441521 (+ 2459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.441726 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.441800 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.441994 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":1035,"threads_started":1}
I20260807 18:21:53.443500 8194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7420566d66434290813037a201798f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:53.446349 7789 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7420566d66434290813037a201798f (DEFAULT_TABLE table=test-table [id=040943adeb9e4ab99ac0881134397a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:53.447559 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.424735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:34889 (local address 127.0.0.1:40474)
0807 18:21:53.425470 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.425491 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.425640 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.439253 (+ 13613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.439259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.439286 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:53.439837 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.439850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.441806 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.441812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.444094 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.444109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.446699 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.446706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.446951 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.446995 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.447065 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.447203 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":421,"threads_started":1}
I20260807 18:21:53.447970 7789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7420566d66434290813037a201798f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:53.448643 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.425026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.29.156.194:34889)
0807 18:21:53.438771 (+ 13745us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.438778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.438802 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:21:53.438880 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.438885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.439084 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.439245 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:53.440050 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.441598 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.444641 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.446409 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.447945 (+ 1536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.448105 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.448170 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.448330 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13592,"thread_start_us":14315,"threads_started":1}
I20260807 18:21:53.452651 7994 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7420566d66434290813037a201798f (DEFAULT_TABLE table=test-table [id=040943adeb9e4ab99ac0881134397a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:53.454246 7994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7420566d66434290813037a201798f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:53.473558 8339 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Bootstrap starting.
I20260807 18:21:53.476770 8342 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Bootstrap starting.
I20260807 18:21:53.479357 8339 tablet_bootstrap.cc:654] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:53.481423 8339 log.cc:824] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:53.481696 8341 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Bootstrap starting.
I20260807 18:21:53.483759 8342 tablet_bootstrap.cc:654] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:53.485409 8342 log.cc:824] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:53.486586 8341 tablet_bootstrap.cc:654] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:53.487468 8339 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: No bootstrap required, opened a new log
I20260807 18:21:53.487835 8339 ts_tablet_manager.cc:1397] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.003s
I20260807 18:21:53.488409 8341 log.cc:824] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Log is configured to *not* fsync() on all Append() calls
I20260807 18:21:53.488722 8342 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: No bootstrap required, opened a new log
I20260807 18:21:53.489147 8342 ts_tablet_manager.cc:1397] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260807 18:21:53.492441 8341 tablet_bootstrap.cc:492] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: No bootstrap required, opened a new log
I20260807 18:21:53.492748 8341 ts_tablet_manager.cc:1397] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260807 18:21:53.499300 8342 raft_consensus.cc:348] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.499938 8342 raft_consensus.cc:374] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:53.500113 8342 raft_consensus.cc:729] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd987eb3710479fb470e4567fa43e0d, State: Initialized, Role: FOLLOWER
I20260807 18:21:53.500897 8342 consensus_queue.cc:260] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.502189 8339 raft_consensus.cc:348] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.502779 8339 raft_consensus.cc:374] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:53.502883 8339 raft_consensus.cc:729] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339595331858460fb21cc29c97346d87, State: Initialized, Role: FOLLOWER
I20260807 18:21:53.503149 8342 ts_tablet_manager.cc:1428] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 18:21:53.503445 8088 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
I20260807 18:21:53.503723 8339 consensus_queue.cc:260] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.509894 8341 raft_consensus.cc:348] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.510377 8341 raft_consensus.cc:374] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:53.510524 8341 raft_consensus.cc:729] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4618ad46e641436b97dcc19e5e85f398, State: Initialized, Role: FOLLOWER
I20260807 18:21:53.511195 8341 consensus_queue.cc:260] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.513653 8341 ts_tablet_manager.cc:1428] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Time spent starting tablet: real 0.021s user 0.009s sys 0.008s
I20260807 18:21:53.513688 8339 ts_tablet_manager.cc:1428] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Time spent starting tablet: real 0.026s user 0.014s sys 0.009s
I20260807 18:21:53.515578 8294 heartbeater.cc:503] Master 127.29.156.254:40821 was elected leader, sending a full tablet report...
I20260807 18:21:53.516830 8347 raft_consensus.cc:482] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:53.517247 8347 raft_consensus.cc:504] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.520257 8347 leader_election.cc:288] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 339595331858460fb21cc29c97346d87 (127.29.156.195:36725), 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:53.529760 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.520316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:34935 (local address 127.29.156.195:36725)
0807 18:21:53.520593 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.520603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.522170 (+ 1567us) server_negotiation.cc:407] Connection header received
0807 18:21:53.522491 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.522498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.522578 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.522631 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:53.523527 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.524827 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.527311 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.528687 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.528857 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.528918 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.529312 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.529415 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:21:53.529904 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.519873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:36725 (local address 127.29.156.194:34935)
0807 18:21:53.522052 (+ 2179us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.522080 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.522298 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.522814 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.522817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.522845 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:53.523291 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.523306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.525014 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.525020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.527133 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.527145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.528902 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.528909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.529127 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.529166 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.529272 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.529411 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2034,"thread_start_us":1825,"threads_started":1}
I20260807 18:21:53.530682 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.522255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:48301 (local address 127.29.156.193:38675)
0807 18:21:53.522484 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.522489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.523174 (+ 685us) server_negotiation.cc:407] Connection header received
0807 18:21:53.523323 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.523328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.523441 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.523586 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:53.524539 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.525670 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.528287 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.529444 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.529609 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.529674 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.530381 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.530485 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:21:53.530946 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.522151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38675 (local address 127.29.156.194:48301)
0807 18:21:53.522981 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:53.522998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:53.523186 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:53.523606 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:53.523613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:53.523645 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:53.524200 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.524215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.525852 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.525857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.528138 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:53.528155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.529784 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.529796 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:53.530183 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.530236 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:21:53.530352 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:21:53.530550 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":482,"threads_started":1}
I20260807 18:21:53.531358 8222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7420566d66434290813037a201798f" candidate_uuid: "7dd987eb3710479fb470e4567fa43e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339595331858460fb21cc29c97346d87" is_pre_election: true
I20260807 18:21:53.532054 8222 raft_consensus.cc:2454] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd987eb3710479fb470e4567fa43e0d in term 0.
I20260807 18:21:53.532133 7819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7420566d66434290813037a201798f" candidate_uuid: "7dd987eb3710479fb470e4567fa43e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4618ad46e641436b97dcc19e5e85f398" is_pre_election: true
I20260807 18:21:53.532701 7819 raft_consensus.cc:2454] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd987eb3710479fb470e4567fa43e0d in term 0.
I20260807 18:21:53.532876 7934 leader_election.cc:302] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339595331858460fb21cc29c97346d87, 7dd987eb3710479fb470e4567fa43e0d; no voters:
I20260807 18:21:53.533571 8347 raft_consensus.cc:2781] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:53.533813 8347 raft_consensus.cc:482] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:53.533931 8347 raft_consensus.cc:3037] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:53.536597 8347 raft_consensus.cc:504] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.537232 8347 leader_election.cc:288] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [CANDIDATE]: Term 1 election: Requested vote from peers 339595331858460fb21cc29c97346d87 (127.29.156.195:36725), 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:53.537849 8222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7420566d66434290813037a201798f" candidate_uuid: "7dd987eb3710479fb470e4567fa43e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339595331858460fb21cc29c97346d87"
I20260807 18:21:53.537950 7819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7420566d66434290813037a201798f" candidate_uuid: "7dd987eb3710479fb470e4567fa43e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4618ad46e641436b97dcc19e5e85f398"
I20260807 18:21:53.538136 8222 raft_consensus.cc:3037] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:53.538334 7819 raft_consensus.cc:3037] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:53.540851 8222 raft_consensus.cc:2454] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd987eb3710479fb470e4567fa43e0d in term 1.
I20260807 18:21:53.541290 7934 leader_election.cc:302] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339595331858460fb21cc29c97346d87, 7dd987eb3710479fb470e4567fa43e0d; no voters:
I20260807 18:21:53.541628 8347 raft_consensus.cc:2781] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:53.541656 7819 raft_consensus.cc:2454] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd987eb3710479fb470e4567fa43e0d in term 1.
I20260807 18:21:53.542344 8347 raft_consensus.cc:686] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 1 LEADER]: Becoming Leader. State: Replica: 7dd987eb3710479fb470e4567fa43e0d, State: Running, Role: LEADER
W20260807 18:21:53.542770 8297 tablet.cc:2369] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:53.543946 8347 consensus_queue.cc:237] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:53.549516 6519 catalog_manager.cc:5495] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194). New cstate: current_term: 1 leader_uuid: "7dd987eb3710479fb470e4567fa43e0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:53.571354 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.564866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.29.156.194:34889)
0807 18:21:53.565062 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.565067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.565085 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:21:53.565174 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.565177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.565253 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.565377 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:53.565861 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.566641 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.568290 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.569070 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.569275 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.569877 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:53.570444 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:53.570560 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.571022 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.571104 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:21:53.581995 7819 raft_consensus.cc:1261] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Refusing update from remote peer 7dd987eb3710479fb470e4567fa43e0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:53.583052 8353 consensus_queue.cc:1035] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:53.583606 8222 raft_consensus.cc:1261] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Refusing update from remote peer 7dd987eb3710479fb470e4567fa43e0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:53.584582 8353 consensus_queue.cc:1035] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:21:53.596475 8090 tablet.cc:2369] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:21:53.604295 8363 mvcc.cc:205] Tried to move back new op lower bound from 7315975838018347008 to 7315975837879914496. Current Snapshot: MvccSnapshot[applied={T|T < 7315975838018347008}]
I20260807 18:21:53.615322 8364 mvcc.cc:205] Tried to move back new op lower bound from 7315975838018347008 to 7315975837879914496. Current Snapshot: MvccSnapshot[applied={T|T < 7315975838018347008}]
I20260807 18:21:53.628419 6518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:53.628716 6518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:53.637233 6518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ce4dfbda84045698f18de062d602c01: Sending DeleteTablet for 3 replicas of tablet 4b7420566d66434290813037a201798f
I20260807 18:21:53.638425 8194 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7420566d66434290813037a201798f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:53 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:53650
I20260807 18:21:53.639679 7994 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7420566d66434290813037a201798f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:53 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:40474
I20260807 18:21:53.639931 8376 tablet_replica.cc:330] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: stopping tablet replica
I20260807 18:21:53.640483 8376 raft_consensus.cc:2229] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:53.640601 7789 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7420566d66434290813037a201798f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:53 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:60376
I20260807 18:21:53.641100 8376 raft_consensus.cc:2258] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:53.644565 8376 ts_tablet_manager.cc:1905] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:53.654361 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 6472
I20260807 18:21:53.654630 8376 ts_tablet_manager.cc:1918] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:53.654830 8376 log.cc:1196] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/4b7420566d66434290813037a201798f
I20260807 18:21:53.655627 8376 ts_tablet_manager.cc:1939] T 4b7420566d66434290813037a201798f P 339595331858460fb21cc29c97346d87: Deleting consensus metadata
I20260807 18:21:53.655905 6559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:53.660331 8378 tablet_replica.cc:330] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: stopping tablet replica
I20260807 18:21:53.660971 8378 raft_consensus.cc:2229] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:53.661669 8378 raft_consensus.cc:2258] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:53.662616 8377 tablet_replica.cc:330] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: stopping tablet replica
I20260807 18:21:53.663003 8377 raft_consensus.cc:2229] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:53.663661 8377 raft_consensus.cc:2258] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:53.665679 8377 ts_tablet_manager.cc:1905] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:53.666411 8378 ts_tablet_manager.cc:1905] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:53.673825 8377 ts_tablet_manager.cc:1918] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:53.674036 8377 log.cc:1196] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/4b7420566d66434290813037a201798f
I20260807 18:21:53.674587 8378 ts_tablet_manager.cc:1918] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:53.674750 8377 ts_tablet_manager.cc:1939] T 4b7420566d66434290813037a201798f P 7dd987eb3710479fb470e4567fa43e0d: Deleting consensus metadata
I20260807 18:21:53.674894 8378 log.cc:1196] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/4b7420566d66434290813037a201798f
I20260807 18:21:53.675729 8378 ts_tablet_manager.cc:1939] T 4b7420566d66434290813037a201798f P 4618ad46e641436b97dcc19e5e85f398: Deleting consensus metadata
I20260807 18:21:53.682143 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.641839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.29.156.253:36855)
0807 18:21:53.642040 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.642047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.652974 (+ 10927us) server_negotiation.cc:407] Connection header received
0807 18:21:53.653122 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.653128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.653221 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.653337 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.659356 (+ 6019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.660803 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.663662 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.664410 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.664578 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.674413 (+ 9835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.674420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.674425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.674474 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.678585 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.680064 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.680069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.680072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.680202 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.680662 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.680670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.680673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.680982 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.681255 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.681592 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.681825 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:21:53.696934 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.652635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.29.156.252:46805)
0807 18:21:53.652908 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.652914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.682230 (+ 29316us) server_negotiation.cc:407] Connection header received
0807 18:21:53.682811 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.682818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.682908 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.682991 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.684903 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.685739 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.687209 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.687778 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.687917 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.690618 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.690624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.690628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.690661 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.693610 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.694404 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.694435 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.694439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.694601 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.695198 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.695202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.695204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.695720 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.696047 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.696317 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.696592 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:21:53.863909 6499 catalog_manager.cc:4841] TS 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675): tablet 4b7420566d66434290813037a201798f (table test-table [id=040943adeb9e4ab99ac0881134397a1d]) successfully deleted
I20260807 18:21:53.864359 6499 catalog_manager.cc:4841] TS 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889): tablet 4b7420566d66434290813037a201798f (table test-table [id=040943adeb9e4ab99ac0881134397a1d]) successfully deleted
I20260807 18:21:53.864619 6499 catalog_manager.cc:4841] TS 339595331858460fb21cc29c97346d87 (127.29.156.195:36725): tablet 4b7420566d66434290813037a201798f (table test-table [id=040943adeb9e4ab99ac0881134397a1d]) successfully deleted
I20260807 18:21:53.867733 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 6472
I20260807 18:21:53.943693 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.918679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52102 (local address 127.29.156.253:36855)
0807 18:21:53.918927 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.918938 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.921192 (+ 2254us) server_negotiation.cc:407] Connection header received
0807 18:21:53.922447 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.922453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.922546 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.922615 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.927159 (+ 4544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.928422 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.931326 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.932151 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.932323 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.935726 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.935734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.935739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.935788 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.940024 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.941012 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.941018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.941024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.941196 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.941728 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.941734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.941738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.942136 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.942400 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.943025 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.943299 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:21:53.945011 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:53.920635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.29.156.252:46805)
0807 18:21:53.920844 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:53.920851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:53.929712 (+ 8861us) server_negotiation.cc:407] Connection header received
0807 18:21:53.930186 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:53.930193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:53.930287 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:53.930360 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:53.932694 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.933947 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.935943 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:53.936366 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:53.936513 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:53.939092 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:53.939097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:53.939100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:53.939136 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:53.941860 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.942573 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.942579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.942584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.942714 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:53.943168 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:53.943172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:53.943175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:53.943566 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:53.943798 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:53.944377 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 18:21:53.944635 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:21:54.614670 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:54.613597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:57715)
0807 18:21:54.614268 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:54.614448 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":344,"threads_started":1}
W20260807 18:21:54.615185 8088 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:40821 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:54.625718 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:54.624720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:52937)
0807 18:21:54.625210 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:54.625370 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":248,"threads_started":1}
W20260807 18:21:54.626503 8294 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:40821 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:54.626951 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:54.625928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:40605)
0807 18:21:54.626578 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:54.626722 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"threads_started":1}
W20260807 18:21:54.627348 7883 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:40821 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.137540 8401 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ce4dfbda84045698f18de062d602c01)
I20260807 18:21:55.137897 8401 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:55.138981 8402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ce4dfbda84045698f18de062d602c01)
I20260807 18:21:55.139274 8402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:55.139437 8401 leader_election.cc:288] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), 3091390305e34940bfce4c7046aafac6 (127.29.156.253:36855)
I20260807 18:21:55.147394 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.139637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39168)
0807 18:21:55.146755 (+ 7118us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.147019 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7006,"thread_start_us":7315,"threads_started":1}
W20260807 18:21:55.148758 7619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.149654 8402 leader_election.cc:288] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252:46805)
I20260807 18:21:55.149904 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.141354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39176)
0807 18:21:55.149083 (+ 7729us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.149281 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7590,"thread_start_us":6385,"threads_started":1}
W20260807 18:21:55.150195 7343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.151401 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.150952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39186)
0807 18:21:55.151111 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.151214 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 18:21:55.151696 7619 leader_election.cc:334] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.152855 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.152435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39194)
0807 18:21:55.152648 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.152736 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 18:21:55.153101 7343 leader_election.cc:334] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.157469 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.147979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:46805 (local address 127.0.0.1:45192)
0807 18:21:55.149175 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.149203 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:55.149324 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:55.150284 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:55.150289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:55.150323 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:55.150827 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:55.150847 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.152546 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.152554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:55.154924 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:55.154938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.156572 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.156579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:55.156766 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:55.156809 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:21:55.156910 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:21:55.157061 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"thread_start_us":825,"threads_started":1}
I20260807 18:21:55.157554 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.139847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:36855 (local address 127.0.0.1:52104)
0807 18:21:55.147325 (+ 7478us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.147350 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:55.147792 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:55.148577 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:55.148582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:55.148614 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:55.149083 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:55.149102 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.152280 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.152287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:55.154712 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:55.154733 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.156417 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.156444 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:55.156709 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:55.156751 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:21:55.156875 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:21:55.157050 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7296,"mutex_wait_us":1,"thread_start_us":6753,"threads_started":1}
I20260807 18:21:55.157564 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.139843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.29.156.253:36855)
0807 18:21:55.146568 (+ 6725us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:55.146578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:55.147624 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 18:21:55.148018 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:55.148026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:55.148112 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:55.148181 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:55.149350 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.152104 (+ 2754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.154900 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.156231 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.156394 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:55.156486 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:55.156949 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:21:55.157098 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6518,"thread_start_us":6296,"threads_started":1}
I20260807 18:21:55.158039 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.148109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.29.156.252:46805)
0807 18:21:55.149569 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:55.149576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:55.149598 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:21:55.149658 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:55.149662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:55.149747 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:55.149849 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:55.151146 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.152386 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.155097 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:55.156370 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:55.156529 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:55.156595 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:55.156948 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 18:21:55.157196 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":454,"thread_start_us":282,"threads_started":1}
I20260807 18:21:55.158354 7381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd7d27bf313545e8bea040a5dabe65ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3091390305e34940bfce4c7046aafac6" is_pre_election: true
I20260807 18:21:55.158629 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3091390305e34940bfce4c7046aafac6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd7d27bf313545e8bea040a5dabe65ef" is_pre_election: true
I20260807 18:21:55.158799 7381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd7d27bf313545e8bea040a5dabe65ef in term 1.
I20260807 18:21:55.158984 7647 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3091390305e34940bfce4c7046aafac6 in term 1.
I20260807 18:21:55.159368 7616 leader_election.cc:302] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3091390305e34940bfce4c7046aafac6, fd7d27bf313545e8bea040a5dabe65ef; no voters: 6ce4dfbda84045698f18de062d602c01
I20260807 18:21:55.159507 7341 leader_election.cc:302] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3091390305e34940bfce4c7046aafac6, fd7d27bf313545e8bea040a5dabe65ef; no voters: 6ce4dfbda84045698f18de062d602c01
I20260807 18:21:55.159835 8401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:21:55.160017 8401 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ce4dfbda84045698f18de062d602c01)
I20260807 18:21:55.160023 8402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:21:55.160144 8401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:21:55.160173 8402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ce4dfbda84045698f18de062d602c01)
I20260807 18:21:55.160270 8402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:21:55.162637 8402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:55.162664 8401 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:55.163316 8402 leader_election.cc:288] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 election: Requested vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), fd7d27bf313545e8bea040a5dabe65ef (127.29.156.252:46805)
I20260807 18:21:55.163401 8401 leader_election.cc:288] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 election: Requested vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), 3091390305e34940bfce4c7046aafac6 (127.29.156.253:36855)
I20260807 18:21:55.163875 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.163451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39220)
0807 18:21:55.163619 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.163723 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 18:21:55.164103 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3091390305e34940bfce4c7046aafac6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd7d27bf313545e8bea040a5dabe65ef"
I20260807 18:21:55.164160 7381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd7d27bf313545e8bea040a5dabe65ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3091390305e34940bfce4c7046aafac6"
I20260807 18:21:55.164494 7647 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3091390305e34940bfce4c7046aafac6 in current term 2: Already voted for candidate fd7d27bf313545e8bea040a5dabe65ef in this term.
I20260807 18:21:55.164500 7381 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fd7d27bf313545e8bea040a5dabe65ef in current term 2: Already voted for candidate 3091390305e34940bfce4c7046aafac6 in this term.
I20260807 18:21:55.165189 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.163457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39208)
0807 18:21:55.164899 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.165014 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1312}
I20260807 18:21:55.165844 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.165352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39230)
0807 18:21:55.165502 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.165649 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 18:21:55.166071 7619 leader_election.cc:334] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.166224 7619 leader_election.cc:302] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd7d27bf313545e8bea040a5dabe65ef; no voters: 3091390305e34940bfce4c7046aafac6, 6ce4dfbda84045698f18de062d602c01
I20260807 18:21:55.166450 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.166009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39246)
0807 18:21:55.166196 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.166292 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:21:55.166568 8401 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 18:21:55.166615 7343 leader_election.cc:334] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:55.166790 7343 leader_election.cc:302] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3091390305e34940bfce4c7046aafac6; no voters: 6ce4dfbda84045698f18de062d602c01, fd7d27bf313545e8bea040a5dabe65ef
I20260807 18:21:55.167160 8402 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 18:21:55.616786 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.616002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:49805)
0807 18:21:55.616486 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.616583 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260807 18:21:55.628340 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.627301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:47781)
0807 18:21:55.627983 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.628114 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":425,"threads_started":1}
I20260807 18:21:55.629333 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:55.628149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:40619)
0807 18:21:55.629083 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:55.629168 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":689,"threads_started":1}
I20260807 18:21:56.618574 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.617842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:39645)
0807 18:21:56.618308 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.618393 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260807 18:21:56.640286 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.630206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:37963)
0807 18:21:56.639972 (+ 9766us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.640097 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9644,"thread_start_us":9529,"threads_started":1}
I20260807 18:21:56.641218 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.629418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:49955)
0807 18:21:56.640957 (+ 11539us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.641053 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11427,"thread_start_us":11307,"threads_started":1}
I20260807 18:21:56.913905 8422 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:56.914245 8422 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:56.914935 8422 leader_election.cc:288] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), 3091390305e34940bfce4c7046aafac6 (127.29.156.253:36855)
I20260807 18:21:56.915678 7381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd7d27bf313545e8bea040a5dabe65ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3091390305e34940bfce4c7046aafac6" is_pre_election: true
I20260807 18:21:56.915966 7381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd7d27bf313545e8bea040a5dabe65ef in term 2.
I20260807 18:21:56.916208 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.915215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39534)
0807 18:21:56.915796 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.915928 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":744,"threads_started":1}
I20260807 18:21:56.916520 7616 leader_election.cc:302] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3091390305e34940bfce4c7046aafac6, fd7d27bf313545e8bea040a5dabe65ef; no voters:
I20260807 18:21:56.916849 8422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 18:21:56.917001 8422 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:56.917140 8422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:21:56.918026 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.917598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39544)
0807 18:21:56.917775 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.917875 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:21:56.918251 7619 leader_election.cc:334] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:56.919689 8422 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:56.920312 8422 leader_election.cc:288] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 election: Requested vote from peers 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821), 3091390305e34940bfce4c7046aafac6 (127.29.156.253:36855)
I20260807 18:21:56.920856 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.920459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39554)
0807 18:21:56.920628 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.920715 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:21:56.920893 7381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd7d27bf313545e8bea040a5dabe65ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3091390305e34940bfce4c7046aafac6"
I20260807 18:21:56.921159 7381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:21:56.922060 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:56.921700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39556)
0807 18:21:56.921865 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:56.921941 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:21:56.922297 7619 leader_election.cc:334] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
I20260807 18:21:56.923597 7381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd7d27bf313545e8bea040a5dabe65ef in term 3.
I20260807 18:21:56.924037 7616 leader_election.cc:302] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3091390305e34940bfce4c7046aafac6, fd7d27bf313545e8bea040a5dabe65ef; no voters: 6ce4dfbda84045698f18de062d602c01
I20260807 18:21:56.924362 8422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:21:56.924873 8422 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [term 3 LEADER]: Becoming Leader. State: Replica: fd7d27bf313545e8bea040a5dabe65ef, State: Running, Role: LEADER
I20260807 18:21:56.925266 8422 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } }
I20260807 18:21:56.927294 8425 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd7d27bf313545e8bea040a5dabe65ef. Latest consensus state: current_term: 3 leader_uuid: "fd7d27bf313545e8bea040a5dabe65ef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:56.927680 8425 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:56.928288 8427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:21:56.931869 8427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:21:56.932734 8427 catalog_manager.cc:1259] Loaded cluster ID: 089e4010c63644dabbfbbb1cc1e97e96
I20260807 18:21:56.932838 8427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:21:56.933987 8427 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:21:56.934939 8427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Loaded TSK: 0
I20260807 18:21:56.935847 8427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:21:56.996093 7634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:21:57.004078 7634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:21:57.009109 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.008572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39572)
0807 18:21:57.008851 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.008940 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 18:21:57.009152 7381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [term 3 FOLLOWER]: Refusing update from remote peer fd7d27bf313545e8bea040a5dabe65ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 18:21:57.010067 8425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 18:21:57.010388 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.009912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39580)
0807 18:21:57.010136 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.010206 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260807 18:21:57.010579 7619 consensus_peers.cc:487] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef -> Peer 6ce4dfbda84045698f18de062d602c01 (127.29.156.254:40821): Couldn't send request to peer 6ce4dfbda84045698f18de062d602c01. Status: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:21:57.013038 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd7d27bf313545e8bea040a5dabe65ef. Latest consensus state: current_term: 3 leader_uuid: "fd7d27bf313545e8bea040a5dabe65ef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:57.013408 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:57.015183 8425 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fd7d27bf313545e8bea040a5dabe65ef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:57.015568 8425 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef [sys.catalog]: This master's current role is: LEADER
I20260807 18:21:57.015904 8430 mvcc.cc:205] Tried to move back new op lower bound from 7315975852060889088 to 7315975851729170432. Current Snapshot: MvccSnapshot[applied={T|T < 7315975852060889088}]
I20260807 18:21:57.016896 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fd7d27bf313545e8bea040a5dabe65ef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ce4dfbda84045698f18de062d602c01" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40821 } } peers { permanent_uuid: "3091390305e34940bfce4c7046aafac6" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 36855 } attrs { promote: false } } peers { permanent_uuid: "fd7d27bf313545e8bea040a5dabe65ef" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 46805 } attrs { promote: false } } }
I20260807 18:21:57.017206 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3091390305e34940bfce4c7046aafac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:21:57.035702 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.024898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.29.156.194:34889)
0807 18:21:57.026315 (+ 1417us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.026323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.026348 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:21:57.026445 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.026453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.026561 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.027877 (+ 1316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:57.028180 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.029704 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.033123 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.034176 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.034369 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.034465 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.035125 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.035281 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":980,"threads_started":1}
I20260807 18:21:57.036809 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.024762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:34889 (local address 127.0.0.1:40496)
0807 18:21:57.024993 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.025017 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:57.025155 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:57.026859 (+ 1704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:57.026867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:57.026906 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:57.027567 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.027585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.029926 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.029936 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.032889 (+ 2953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.032905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.034635 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.034642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.034895 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.034942 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 18:21:57.036254 (+ 1312us) client_negotiation.cc:240] Negotiation successful
0807 18:21:57.036448 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 18:21:57.037751 7994 tablet_service.cc:1467] Processing CreateTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 (DEFAULT_TABLE table=test-table [id=a9c1c1a560274afc989ebb534b477ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:57.038300 7994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a10fcfeca024992a7ef548e9d3b69b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:57.040271 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.025448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.29.156.195:36725)
0807 18:21:57.033032 (+ 7584us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.033041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.033679 (+ 638us) server_negotiation.cc:407] Connection header received
0807 18:21:57.033859 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.033865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.033948 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.034828 (+ 880us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:57.035051 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.036179 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.038437 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.039250 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.039442 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.039518 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.039808 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.039961 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7393,"thread_start_us":7240,"threads_started":1}
I20260807 18:21:57.040266 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.025378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:36725 (local address 127.0.0.1:53666)
0807 18:21:57.033542 (+ 8164us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.033575 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:57.033713 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:57.034113 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:57.034118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:57.034150 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:57.034642 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.034654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.036352 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.036357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.038217 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.038230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.039452 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.039459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.039645 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.039670 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:21:57.039785 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:21:57.039910 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8018,"thread_start_us":7869,"threads_started":1}
I20260807 18:21:57.041425 8194 tablet_service.cc:1467] Processing CreateTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 (DEFAULT_TABLE table=test-table [id=a9c1c1a560274afc989ebb534b477ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:57.041931 8194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a10fcfeca024992a7ef548e9d3b69b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:57.042495 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.034338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38675 (local address 127.0.0.1:60386)
0807 18:21:57.035021 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.035035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:57.035173 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:57.035805 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:57.035809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:57.035837 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:57.036281 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.036292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.037709 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.037713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.039839 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.039858 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.041689 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.041695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.041873 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.041917 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:21:57.042024 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:21:57.042159 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":412,"threads_started":1}
I20260807 18:21:57.042549 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.033814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.29.156.193:38675)
0807 18:21:57.034347 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.034354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.035126 (+ 772us) server_negotiation.cc:407] Connection header received
0807 18:21:57.035308 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.035313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.035434 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.035570 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:57.036522 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.037522 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.040022 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.041273 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.041409 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.041496 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.042166 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.042290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":280,"threads_started":1}
I20260807 18:21:57.043633 7789 tablet_service.cc:1467] Processing CreateTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 (DEFAULT_TABLE table=test-table [id=a9c1c1a560274afc989ebb534b477ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:21:57.044106 7789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a10fcfeca024992a7ef548e9d3b69b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:21:57.048924 8443 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Bootstrap starting.
I20260807 18:21:57.050602 8446 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Bootstrap starting.
I20260807 18:21:57.050976 8442 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Bootstrap starting.
I20260807 18:21:57.051292 8443 tablet_bootstrap.cc:654] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:57.052569 8446 tablet_bootstrap.cc:654] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:57.053155 8442 tablet_bootstrap.cc:654] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Neither blocks nor log segments found. Creating new log.
I20260807 18:21:57.053323 8443 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: No bootstrap required, opened a new log
I20260807 18:21:57.053496 8443 ts_tablet_manager.cc:1397] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:21:57.054510 8443 raft_consensus.cc:348] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.054780 8443 raft_consensus.cc:374] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:57.054975 8443 raft_consensus.cc:729] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339595331858460fb21cc29c97346d87, State: Initialized, Role: FOLLOWER
I20260807 18:21:57.055130 8446 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: No bootstrap required, opened a new log
I20260807 18:21:57.055320 8446 ts_tablet_manager.cc:1397] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 18:21:57.055380 8443 consensus_queue.cc:260] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.056584 8443 ts_tablet_manager.cc:1428] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:21:57.056478 8446 raft_consensus.cc:348] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.056981 8442 tablet_bootstrap.cc:492] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: No bootstrap required, opened a new log
I20260807 18:21:57.056964 8446 raft_consensus.cc:374] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:57.057091 8446 raft_consensus.cc:729] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4618ad46e641436b97dcc19e5e85f398, State: Initialized, Role: FOLLOWER
I20260807 18:21:57.057152 8442 ts_tablet_manager.cc:1397] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:21:57.057379 8446 consensus_queue.cc:260] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.057767 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.057246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:37807)
0807 18:21:57.057515 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.057619 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260807 18:21:57.057982 8442 raft_consensus.cc:348] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.058225 8442 raft_consensus.cc:374] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:21:57.058295 8442 raft_consensus.cc:729] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd987eb3710479fb470e4567fa43e0d, State: Initialized, Role: FOLLOWER
I20260807 18:21:57.058523 8442 consensus_queue.cc:260] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.060249 8446 ts_tablet_manager.cc:1428] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260807 18:21:57.061684 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.060942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:50589)
0807 18:21:57.061366 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.061478 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260807 18:21:57.064378 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.063871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:54761)
0807 18:21:57.064092 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.064191 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 18:21:57.064649 7882 heartbeater.cc:503] Master 127.29.156.252:46805 was elected leader, sending a full tablet report...
I20260807 18:21:57.067008 8454 raft_consensus.cc:482] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:21:57.067237 8454 raft_consensus.cc:504] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.068567 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.067915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:39191)
0807 18:21:57.068230 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.068344 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 18:21:57.068667 8292 heartbeater.cc:503] Master 127.29.156.252:46805 was elected leader, sending a full tablet report...
I20260807 18:21:57.069469 8454 leader_election.cc:288] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889), 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:57.071081 8442 ts_tablet_manager.cc:1428] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Time spent starting tablet: real 0.014s user 0.004s sys 0.007s
I20260807 18:21:57.071285 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.070681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:51175)
0807 18:21:57.070930 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.071025 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 18:21:57.071746 8087 heartbeater.cc:503] Master 127.29.156.252:46805 was elected leader, sending a full tablet report...
I20260807 18:21:57.072540 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.072094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:34507)
0807 18:21:57.072275 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.072367 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:21:57.077849 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.069947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:35409 (local address 127.29.156.194:34889)
0807 18:21:57.070100 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.070106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.070128 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:21:57.070211 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.070218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.070300 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.070414 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:57.072291 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.073411 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.075954 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.076893 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.077087 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.077164 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.077473 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.077603 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:21:57.077869 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.069802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:34889 (local address 127.29.156.195:35409)
0807 18:21:57.069932 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.069950 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:57.070081 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:57.071632 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:57.071636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:57.071663 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:57.071937 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.071954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.073557 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.073562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.075746 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.075764 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.077122 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.077128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.077308 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.077334 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:21:57.077447 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:21:57.077590 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 18:21:57.079032 8026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a10fcfeca024992a7ef548e9d3b69b3" candidate_uuid: "339595331858460fb21cc29c97346d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd987eb3710479fb470e4567fa43e0d" is_pre_election: true
I20260807 18:21:57.079491 8026 raft_consensus.cc:2454] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339595331858460fb21cc29c97346d87 in term 0.
I20260807 18:21:57.080190 8132 leader_election.cc:302] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339595331858460fb21cc29c97346d87, 7dd987eb3710479fb470e4567fa43e0d; no voters:
I20260807 18:21:57.080679 8454 raft_consensus.cc:2781] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:21:57.080821 8454 raft_consensus.cc:482] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:21:57.080945 8454 raft_consensus.cc:3037] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:57.082710 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.073900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38675 (local address 127.29.156.195:38595)
0807 18:21:57.076319 (+ 2419us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.076337 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:21:57.076522 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:21:57.076858 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:21:57.076862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:21:57.076886 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:21:57.077189 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.077198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.078406 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.078413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.080286 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:21:57.080303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.081954 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.081960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:21:57.082124 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.082150 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:21:57.082237 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:21:57.082390 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2281,"thread_start_us":2058,"threads_started":1}
I20260807 18:21:57.082738 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.073960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:38595 (local address 127.29.156.193:38675)
0807 18:21:57.074174 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.074182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.076419 (+ 2237us) server_negotiation.cc:407] Connection header received
0807 18:21:57.076662 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.076667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.076741 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.076841 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:21:57.077346 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.078273 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.080451 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.081759 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.081924 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.082013 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.082313 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.082460 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:21:57.083782 7817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a10fcfeca024992a7ef548e9d3b69b3" candidate_uuid: "339595331858460fb21cc29c97346d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4618ad46e641436b97dcc19e5e85f398" is_pre_election: true
I20260807 18:21:57.084200 7817 raft_consensus.cc:2454] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339595331858460fb21cc29c97346d87 in term 0.
I20260807 18:21:57.084228 8454 raft_consensus.cc:504] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.084874 8454 leader_election.cc:288] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889), 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675)
I20260807 18:21:57.085690 7817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a10fcfeca024992a7ef548e9d3b69b3" candidate_uuid: "339595331858460fb21cc29c97346d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4618ad46e641436b97dcc19e5e85f398"
I20260807 18:21:57.085721 8026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a10fcfeca024992a7ef548e9d3b69b3" candidate_uuid: "339595331858460fb21cc29c97346d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd987eb3710479fb470e4567fa43e0d"
I20260807 18:21:57.085875 7817 raft_consensus.cc:3037] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:57.085981 8026 raft_consensus.cc:3037] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:21:57.088528 7817 raft_consensus.cc:2454] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339595331858460fb21cc29c97346d87 in term 1.
I20260807 18:21:57.088886 8026 raft_consensus.cc:2454] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339595331858460fb21cc29c97346d87 in term 1.
I20260807 18:21:57.088922 8132 leader_election.cc:302] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339595331858460fb21cc29c97346d87, 4618ad46e641436b97dcc19e5e85f398; no voters:
I20260807 18:21:57.089208 8454 raft_consensus.cc:2781] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:21:57.089393 8454 raft_consensus.cc:686] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 1 LEADER]: Becoming Leader. State: Replica: 339595331858460fb21cc29c97346d87, State: Running, Role: LEADER
I20260807 18:21:57.089807 8454 consensus_queue.cc:237] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } }
I20260807 18:21:57.090728 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.090284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:36045)
0807 18:21:57.090425 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.090540 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 18:21:57.093432 7635 catalog_manager.cc:5495] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 339595331858460fb21cc29c97346d87 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "339595331858460fb21cc29c97346d87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "339595331858460fb21cc29c97346d87" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 36725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 } health_report { overall_health: UNKNOWN } } }
I20260807 18:21:57.117048 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.111665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.29.156.195:36725)
0807 18:21:57.111896 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.111900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.111915 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:21:57.112019 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.112023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.112107 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.112268 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:21:57.112631 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.113343 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.114805 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.115320 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.115477 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.115918 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:21:57.116440 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:21:57.116618 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.116700 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.116782 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:21:57.121279 7817 raft_consensus.cc:1261] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Refusing update from remote peer 339595331858460fb21cc29c97346d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:57.121502 8026 raft_consensus.cc:1261] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Refusing update from remote peer 339595331858460fb21cc29c97346d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:21:57.122095 8454 consensus_queue.cc:1035] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4618ad46e641436b97dcc19e5e85f398" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:57.122442 8451 consensus_queue.cc:1035] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd987eb3710479fb470e4567fa43e0d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:21:57.124874 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.124347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:47845)
0807 18:21:57.124559 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.124674 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:21:57.126773 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.126256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:43777)
0807 18:21:57.126486 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.126588 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 18:21:57.132525 8461 mvcc.cc:205] Tried to move back new op lower bound from 7315975852520673280 to 7315975852404150272. Current Snapshot: MvccSnapshot[applied={T|T < 7315975852520673280}]
I20260807 18:21:57.134464 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.133990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.194:56645)
0807 18:21:57.134207 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.134300 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 18:21:57.135285 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.131850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:37355)
0807 18:21:57.134966 (+ 3116us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.135067 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 18:21:57.141018 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.140238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.193:59851)
0807 18:21:57.140607 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.140702 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261}
I20260807 18:21:57.148398 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.147817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.29.156.195:44695)
0807 18:21:57.148097 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.148198 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260807 18:21:57.148542 7635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:57.148797 7635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:21:57.155352 7635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd7d27bf313545e8bea040a5dabe65ef: Sending DeleteTablet for 3 replicas of tablet 2a10fcfeca024992a7ef548e9d3b69b3
I20260807 18:21:57.156745 8194 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:57 UTC) from {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:53666
I20260807 18:21:57.157410 8193 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:57 UTC) from {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:53666
I20260807 18:21:57.157881 7994 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:57 UTC) from {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:40496
I20260807 18:21:57.158413 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 7601
I20260807 18:21:57.158380 7789 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a10fcfeca024992a7ef548e9d3b69b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:21:57 UTC) from {username='oryx', principal='oryx/127.29.156.252@KRBTEST.COM'} at 127.0.0.1:60386
I20260807 18:21:57.165712 8470 tablet_replica.cc:330] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: stopping tablet replica
I20260807 18:21:57.166116 8470 raft_consensus.cc:2229] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:57.166640 8470 raft_consensus.cc:2258] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:57.172573 8470 ts_tablet_manager.cc:1905] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:57.172753 8471 tablet_replica.cc:330] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: stopping tablet replica
I20260807 18:21:57.173007 8471 raft_consensus.cc:2229] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:21:57.173367 8471 raft_consensus.cc:2258] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:57.174765 8471 ts_tablet_manager.cc:1905] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:57.175819 8469 tablet_replica.cc:330] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: stopping tablet replica
I20260807 18:21:57.176141 8469 raft_consensus.cc:2229] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:21:57.176852 8469 raft_consensus.cc:2258] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:21:57.178107 8469 ts_tablet_manager.cc:1905] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:21:57.181133 8470 ts_tablet_manager.cc:1918] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:57.181363 8470 log.cc:1196] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/2a10fcfeca024992a7ef548e9d3b69b3
I20260807 18:21:57.181969 8470 ts_tablet_manager.cc:1939] T 2a10fcfeca024992a7ef548e9d3b69b3 P 7dd987eb3710479fb470e4567fa43e0d: Deleting consensus metadata
I20260807 18:21:57.182487 8471 ts_tablet_manager.cc:1918] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:57.182688 8471 log.cc:1196] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/2a10fcfeca024992a7ef548e9d3b69b3
I20260807 18:21:57.183192 8471 ts_tablet_manager.cc:1939] T 2a10fcfeca024992a7ef548e9d3b69b3 P 4618ad46e641436b97dcc19e5e85f398: Deleting consensus metadata
I20260807 18:21:57.184072 7618 catalog_manager.cc:4841] TS 7dd987eb3710479fb470e4567fa43e0d (127.29.156.194:34889): tablet 2a10fcfeca024992a7ef548e9d3b69b3 (table test-table [id=a9c1c1a560274afc989ebb534b477ebf]) successfully deleted
I20260807 18:21:57.184680 7618 catalog_manager.cc:4841] TS 4618ad46e641436b97dcc19e5e85f398 (127.29.156.193:38675): tablet 2a10fcfeca024992a7ef548e9d3b69b3 (table test-table [id=a9c1c1a560274afc989ebb534b477ebf]) successfully deleted
I20260807 18:21:57.185612 8469 ts_tablet_manager.cc:1918] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:21:57.185756 8469 log.cc:1196] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/2a10fcfeca024992a7ef548e9d3b69b3
I20260807 18:21:57.186266 8469 ts_tablet_manager.cc:1939] T 2a10fcfeca024992a7ef548e9d3b69b3 P 339595331858460fb21cc29c97346d87: Deleting consensus metadata
I20260807 18:21:57.187500 7618 catalog_manager.cc:4841] TS 339595331858460fb21cc29c97346d87 (127.29.156.195:36725): tablet 2a10fcfeca024992a7ef548e9d3b69b3 (table test-table [id=a9c1c1a560274afc989ebb534b477ebf]) successfully deleted
W20260807 18:21:57.188174 7618 catalog_manager.cc:4808] TS 339595331858460fb21cc29c97346d87 (127.29.156.195:36725): delete failed for tablet 2a10fcfeca024992a7ef548e9d3b69b3 because the tablet was not found. No further retry: Not found: Tablet not found: 2a10fcfeca024992a7ef548e9d3b69b3
I20260807 18:21:57.193500 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.160821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.29.156.193:38675)
0807 18:21:57.161092 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.161101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.161962 (+ 861us) server_negotiation.cc:407] Connection header received
0807 18:21:57.162308 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.162317 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.162439 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.162531 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:57.167130 (+ 4599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.168934 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.178776 (+ 9842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.179449 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.179602 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.187133 (+ 7531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:57.187150 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:57.187165 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:57.187232 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:57.190455 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.191343 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.191351 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.191353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.191471 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.191931 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.191935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.191937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.192356 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:57.192584 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.192967 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.193176 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:21:57.194185 7873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:57.292552 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 7601
I20260807 18:21:57.325486 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 7900
I20260807 18:21:57.343370 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.326655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.29.156.194:34889)
0807 18:21:57.326852 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.326860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.327385 (+ 525us) server_negotiation.cc:407] Connection header received
0807 18:21:57.327577 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.327583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.327665 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.327763 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:57.329646 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.330843 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.332750 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.333415 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.333563 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.336306 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:57.336326 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:57.336347 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:57.336394 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:57.340295 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.340996 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.341094 (+ 98us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.341098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.341262 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.341640 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.341647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.341651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.342153 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:57.342472 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.342693 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.342939 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:21:57.344362 8082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:57.475625 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 7900
I20260807 18:21:57.483505 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.482975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39590)
0807 18:21:57.483185 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.483299 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:21:57.485037 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.484539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40821 (local address 127.0.0.1:39600)
0807 18:21:57.484730 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:21:57.484837 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:40821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:21:57.508857 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 8106
I20260807 18:21:57.526129 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:21:57.509762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53686 (local address 127.29.156.195:36725)
0807 18:21:57.510084 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:21:57.510092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:21:57.510117 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:21:57.510195 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:21:57.510199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:21:57.510286 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:21:57.510412 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:21:57.511942 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.512815 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.514237 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:21:57.514713 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:21:57.514893 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:21:57.518875 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:21:57.518893 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:21:57.518910 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:21:57.518951 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:21:57.522356 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.523298 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.523306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.523308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.523468 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:21:57.524240 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:21:57.524245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:21:57.524248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:21:57.524745 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:21:57.525022 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 18:21:57.525503 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 18:21:57.525693 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:21:57.526957 8285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:57.665382 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 8106
I20260807 18:21:57.697930 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 7146
I20260807 18:21:57.699324 7409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:57.820644 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 7146
I20260807 18:21:57.848758 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 7305
I20260807 18:21:57.849961 7672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:21:57.977084 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 7305
2026-08-07T18:21:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12311 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:21:58 dist-test-slave-sb2z krb5kdc[8483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:21:58 dist-test-slave-sb2z krb5kdc[8483](info): set up 2 sockets
Aug 07 18:21:58 dist-test-slave-sb2z krb5kdc[8483](info): commencing operation
krb5kdc: starting...
W20260807 18:21:59.294966 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:21:59 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:21:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:21:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab.
Entry for principal oryx/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab.
I20260807 18:21:59.603263 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36485
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab
--principal=oryx/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:36485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:21:59.959980 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:21:59.960340 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:21:59.960506 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:21:59.967900 8502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:21:59.968072 8502 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:21:59.968189 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:21:59.968307 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:21:59.968394 8502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:21:59.968485 8502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:21:59.978646 8502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:36485
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.254.keytab
--principal=oryx/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:21:59.980031 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:21:59.981459 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:21:59.993873 8507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:59.994000 8508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:21:59.994045 8510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:21:59.995623 8502 server_base.cc:1034] running on GCE node
I20260807 18:21:59.996490 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:21:59.998190 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:21:59.999461 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786126919999375 us; error 41 us; skew 500 ppm
I20260807 18:21:59.999855 8502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:21:59 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126919, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:00.004043 8502 init.cc:377] Logged in from keytab as oryx/127.29.156.254@KRBTEST.COM (short username oryx)
I20260807 18:22:00.005690 8502 webserver.cc:460] Webserver started at http://127.29.156.254:34711/ using document root <none> and password file <none>
I20260807 18:22:00.006453 8502 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:00.006553 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:00.006855 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:00.010174 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "55a735ba83ef479a8ecf2c137edda320"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "1d13ec8151a167d85812a4558a2d4456"
server_key_iv: "a19c8f1491d91753c997109aa773661b"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.011022 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "55a735ba83ef479a8ecf2c137edda320"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "1d13ec8151a167d85812a4558a2d4456"
server_key_iv: "a19c8f1491d91753c997109aa773661b"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.016764 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:22:00.020705 8519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:00.022414 8502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:22:00.022635 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "55a735ba83ef479a8ecf2c137edda320"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "1d13ec8151a167d85812a4558a2d4456"
server_key_iv: "a19c8f1491d91753c997109aa773661b"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.022853 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:00.057855 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:00.062647 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:00.062889 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:00.085925 8502 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:36485
I20260807 18:22:00.086133 8607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:36485 every 8 connection(s)
I20260807 18:22:00.087363 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:00.091182 8610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:00.094974 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 8502
I20260807 18:22:00.095237 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:22:00.096239 30323 external_mini_cluster.cc:1428] Setting key 3739c6ab7b8b4df272388e7fa0076e7c
I20260807 18:22:00.105930 8610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Bootstrap starting.
I20260807 18:22:00.110056 8610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:00.111491 8610 log.cc:824] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:22:00 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:00.115011 8610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: No bootstrap required, opened a new log
I20260807 18:22:00.122012 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:00.098673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.29.156.254:36485)
0807 18:22:00.099615 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:00.099631 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:00.099685 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:22:00.101591 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:00.101618 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:00.102066 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:00.102433 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:00.104316 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.105767 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.107626 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.108393 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.108564 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:00.113866 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:00.113874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:00.113895 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:00.113935 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:00.117504 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.118318 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.118337 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.118343 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.118530 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.118966 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.118971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.118985 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.119631 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:00.119984 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:00.120419 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:22:00.120917 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":415,"threads_started":1}
I20260807 18:22:00.124702 8610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } }
I20260807 18:22:00.125197 8610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:00.125327 8610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a735ba83ef479a8ecf2c137edda320, State: Initialized, Role: FOLLOWER
I20260807 18:22:00.125953 8610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } }
I20260807 18:22:00.126183 8610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:00.126362 8610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:00.126603 8610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:00.129441 8610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } }
I20260807 18:22:00.129997 8610 leader_election.cc:302] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a735ba83ef479a8ecf2c137edda320; no voters:
I20260807 18:22:00.130661 8610 leader_election.cc:288] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:22:00.130825 8616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:00.131963 8616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [term 1 LEADER]: Becoming Leader. State: Replica: 55a735ba83ef479a8ecf2c137edda320, State: Running, Role: LEADER
I20260807 18:22:00.133102 8610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:00.133590 8616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } }
I20260807 18:22:00.138114 8617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a735ba83ef479a8ecf2c137edda320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } } }
I20260807 18:22:00.138159 8618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a735ba83ef479a8ecf2c137edda320. Latest consensus state: current_term: 1 leader_uuid: "55a735ba83ef479a8ecf2c137edda320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a735ba83ef479a8ecf2c137edda320" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36485 } } }
I20260807 18:22:00.138751 8618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:00.138859 8617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320 [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:00.141599 8632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:00.146365 8632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:00.155632 8632 catalog_manager.cc:1347] Generated new cluster ID: c1c4fa179d0a4a1a85c48f24acacaf1f
I20260807 18:22:00.155771 8632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:00.209067 8632 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:22:00.210570 8632 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:00.222177 8632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Generated new TSK 0
I20260807 18:22:00.223099 8632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
I20260807 18:22:00.315199 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36485
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:00.681311 8641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:00.681617 8641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:00.681797 8641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:00.689381 8641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:00.689503 8641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:00.689656 8641 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:22:00.689783 8641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:00.699738 8641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:00.701227 8641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:00.702673 8641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:00.716802 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:00.716958 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:00.716919 8652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:00.718156 8641 server_base.cc:1034] running on GCE node
I20260807 18:22:00.718869 8641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:00.719974 8641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:00.721240 8641 hybrid_clock.cc:648] HybridClock initialized: now 1786126920721166 us; error 38 us; skew 500 ppm
I20260807 18:22:00.721657 8641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:00 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126920, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:00.725279 8641 init.cc:377] Logged in from keytab as oryx/127.29.156.193@KRBTEST.COM (short username oryx)
I20260807 18:22:00.727291 8641 webserver.cc:460] Webserver started at http://127.29.156.193:34827/ using document root <none> and password file <none>
I20260807 18:22:00.728307 8641 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:00.728433 8641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:00.728758 8641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:00.732088 8641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "c0a59f504de14d1699431f305215cada"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "f9cd56584472571f0528579a57a1d19a"
server_key_iv: "ee2cdaa281654d85472c89047717e1d4"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.733004 8641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "c0a59f504de14d1699431f305215cada"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "f9cd56584472571f0528579a57a1d19a"
server_key_iv: "ee2cdaa281654d85472c89047717e1d4"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.738982 8641 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:22:00.742861 8660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:00.744514 8641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:00.744683 8641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "c0a59f504de14d1699431f305215cada"
format_stamp: "Formatted at 2026-08-07 18:22:00 on dist-test-slave-sb2z"
server_key: "f9cd56584472571f0528579a57a1d19a"
server_key_iv: "ee2cdaa281654d85472c89047717e1d4"
server_key_version: "encryptionkey@0"
I20260807 18:22:00.744897 8641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:00.759959 8641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:00.764624 8641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:00.764864 8641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:00.767797 8641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:00.767885 8641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:00.768049 8641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:00.768119 8641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:22:00 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126920, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:00.795305 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:00.768453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:47145 (local address 127.29.156.254:36485)
0807 18:22:00.769260 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:00.769267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:00.770479 (+ 1212us) server_negotiation.cc:407] Connection header received
0807 18:22:00.773364 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:00.773374 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:00.773505 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:00.773651 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:00.776606 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.777859 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.780022 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.780954 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.781220 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:00.787036 (+ 5816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:00.787048 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:00.787055 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:00.787113 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:00.790795 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.791789 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.791804 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.791807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.791987 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.792771 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.792780 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.792786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.793231 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:00.793397 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:00.794812 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0807 18:22:00.795020 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":553,"threads_started":1}
I20260807 18:22:00.797473 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:00.769264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.193:47145)
0807 18:22:00.770273 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:00.770327 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:00.772877 (+ 2550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:00.774005 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:00.774022 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:00.775010 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:00.776339 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:00.776377 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.778069 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.778073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:00.779772 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:00.779797 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.781340 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.781348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:00.781658 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:00.782875 (+ 1217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:00.782904 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:00.786690 (+ 3786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:00.790988 (+ 4298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:00.791000 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:00.791027 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:00.791630 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:00.792146 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:00.792152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:00.792156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:00.792574 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:00.793495 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:00.793508 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:00.794210 (+ 702us) client_negotiation.cc:769] Sending connection context
0807 18:22:00.794644 (+ 434us) client_negotiation.cc:240] Negotiation successful
0807 18:22:00.795427 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":440,"threads_started":1}
I20260807 18:22:00.823787 8641 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:45739
I20260807 18:22:00.823793 8828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:45739 every 8 connection(s)
I20260807 18:22:00.825451 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:00.806146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.193:38361)
0807 18:22:00.806922 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:00.806945 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:00.807155 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:00.807669 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:00.807676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:00.808507 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:00.809127 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:00.809145 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.810991 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.811000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:00.813267 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:00.813291 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.814508 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.814513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:00.814663 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:00.815512 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:00.815579 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:00.817013 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:00.821935 (+ 4922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:00.821939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:00.821942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:00.822466 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:00.822996 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:00.822999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:00.823002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:00.823245 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:00.824096 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:00.824102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:00.824340 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 18:22:00.824484 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 18:22:00.824889 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":351,"threads_started":1}
I20260807 18:22:00.825716 8641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:00.826813 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:00.806214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38361 (local address 127.29.156.254:36485)
0807 18:22:00.806422 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:00.806426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:00.807083 (+ 657us) server_negotiation.cc:407] Connection header received
0807 18:22:00.807329 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:00.807340 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:00.807450 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:00.807605 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:00.809337 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.810807 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.813477 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:00.814272 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:00.814424 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:00.817309 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:00.817320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:00.817330 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:00.817388 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:00.821795 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.822629 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.822639 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.822644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.822849 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:00.823398 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:00.823407 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:00.823413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:00.823907 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:00.825782 (+ 1875us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:00.825937 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:22:00.826336 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:22:00.828435 8829 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36485
I20260807 18:22:00.828707 8829 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:00.829427 8829 heartbeater.cc:511] Master 127.29.156.254:36485 requested a full tablet report, sending...
I20260807 18:22:00.830428 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 8641
I20260807 18:22:00.830646 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:22:00.831600 30323 external_mini_cluster.cc:1428] Setting key d3e77c726e587d352f027db07d8bfbb0
I20260807 18:22:00.831843 8546 ts_manager.cc:194] Registered new tserver with Master: c0a59f504de14d1699431f305215cada (127.29.156.193:45739)
I20260807 18:22:00.834286 8546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:47145
WARNING: no policy specified for oryx/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
I20260807 18:22:00.898854 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36485
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:01.251466 8833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:01.251863 8833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:01.252056 8833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:01.259419 8833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:01.259620 8833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:01.259778 8833 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:22:01.259929 8833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:01.270026 8833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:01.271435 8833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:01.272801 8833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:01.287030 8839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:01.287220 8843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:01.287256 8841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:01.288034 8833 server_base.cc:1034] running on GCE node
I20260807 18:22:01.288789 8833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:01.289885 8833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:01.291205 8833 hybrid_clock.cc:648] HybridClock initialized: now 1786126921291130 us; error 48 us; skew 500 ppm
I20260807 18:22:01.291671 8833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:01 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126921, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:01.295415 8833 init.cc:377] Logged in from keytab as oryx/127.29.156.194@KRBTEST.COM (short username oryx)
I20260807 18:22:01.297405 8833 webserver.cc:460] Webserver started at http://127.29.156.194:40929/ using document root <none> and password file <none>
I20260807 18:22:01.298242 8833 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:01.298396 8833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:01.298770 8833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:01.302356 8833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "2cd4c058ecc9478c928c4edb6733ca93"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "66d0e8facb786f1fd052aed594a567d3"
server_key_iv: "6ed5e5ac112c075a30e76d664362b6f4"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.303355 8833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "2cd4c058ecc9478c928c4edb6733ca93"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "66d0e8facb786f1fd052aed594a567d3"
server_key_iv: "6ed5e5ac112c075a30e76d664362b6f4"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.309314 8833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:22:01.313482 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:01.314957 8833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:22:01.315202 8833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "2cd4c058ecc9478c928c4edb6733ca93"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "66d0e8facb786f1fd052aed594a567d3"
server_key_iv: "6ed5e5ac112c075a30e76d664362b6f4"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.315476 8833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:01.334049 8833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:01.339375 8833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:01.339735 8833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:01.342731 8833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:01.342862 8833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:01.343048 8833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:01.343155 8833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:22:01 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126921, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:01.371568 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.343446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:32783 (local address 127.29.156.254:36485)
0807 18:22:01.344195 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:01.344202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:01.345500 (+ 1298us) server_negotiation.cc:407] Connection header received
0807 18:22:01.349308 (+ 3808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:01.349318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:01.349472 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:01.349601 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:01.353196 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.354255 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.358184 (+ 3929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.358666 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.358841 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.363348 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:01.363354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:01.363357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:01.363394 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:01.366815 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.368045 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.368054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.368060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.368269 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.368823 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.368831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.368835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.369267 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:01.369427 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:01.370803 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0807 18:22:01.371090 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":423,"threads_started":1}
I20260807 18:22:01.373440 8866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.344193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.194:32783)
0807 18:22:01.345256 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:01.345328 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:01.348728 (+ 3400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:01.350186 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:01.350205 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:01.351430 (+ 1225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:01.352863 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.352898 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.354495 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.354507 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.357949 (+ 3442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.357972 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.358956 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.358964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.359240 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.360187 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:01.360212 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:01.363038 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:01.366981 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.366990 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.367019 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.367819 (+ 800us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.368414 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.368419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.368423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.368677 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.369466 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:01.369479 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:01.370196 (+ 717us) client_negotiation.cc:769] Sending connection context
0807 18:22:01.370681 (+ 485us) client_negotiation.cc:240] Negotiation successful
0807 18:22:01.371346 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":390,"threads_started":1}
I20260807 18:22:01.393128 8833 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:41485
I20260807 18:22:01.393131 9026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:41485 every 8 connection(s)
I20260807 18:22:01.394934 8833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:01.397601 9028 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36485
I20260807 18:22:01.397867 9028 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:01.398746 9028 heartbeater.cc:511] Master 127.29.156.254:36485 requested a full tablet report, sending...
I20260807 18:22:01.400153 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.382795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.194:42777)
0807 18:22:01.383477 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:01.383494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:01.383702 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:01.384120 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:01.384130 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:01.384978 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:01.385580 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.385610 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.387191 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.387197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.389157 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.389180 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.390430 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.390445 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.390690 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.391532 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:01.391586 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:01.393075 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:01.397078 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.397083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.397086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.397584 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.398039 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.398044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.398048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.398202 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.398985 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:01.398991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:01.399162 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 18:22:01.399288 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:22:01.399656 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":381,"threads_started":1}
I20260807 18:22:01.400162 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.382707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:42777 (local address 127.29.156.254:36485)
0807 18:22:01.382905 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:01.382910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:01.383620 (+ 710us) server_negotiation.cc:407] Connection header received
0807 18:22:01.383873 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:01.383879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:01.383964 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:01.384083 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:01.385777 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.387032 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.389351 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.390165 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.390321 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.393345 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:01.393352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:01.393358 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:01.393400 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:01.396956 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.397742 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.397747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.397750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.397906 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.398338 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.398344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.398347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.398808 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:01.399017 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:01.399463 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 18:22:01.399755 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:22:01.400164 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 8833
I20260807 18:22:01.400334 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:22:01.400561 8546 ts_manager.cc:194] Registered new tserver with Master: 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485)
I20260807 18:22:01.400920 30323 external_mini_cluster.cc:1428] Setting key 4cfac2d0e1524535fa7884ffbe8f4df9
I20260807 18:22:01.401448 8546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:32783
WARNING: no policy specified for oryx/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
I20260807 18:22:01.467262 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:36485
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:01.821815 9034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:01.822149 9034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:01.822328 9034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:01.829988 9034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:01.830147 9034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:01.830295 9034 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:22:01.830433 9034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:01.838132 8829 heartbeater.cc:503] Master 127.29.156.254:36485 was elected leader, sending a full tablet report...
I20260807 18:22:01.842136 9034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.156.254:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:01.843676 9034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:01.845177 9034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:01.860754 9041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:01.860692 9042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:01.860716 9045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:01.861738 9034 server_base.cc:1034] running on GCE node
I20260807 18:22:01.862526 9034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:01.863653 9034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:01.864935 9034 hybrid_clock.cc:648] HybridClock initialized: now 1786126921864845 us; error 41 us; skew 500 ppm
I20260807 18:22:01.865343 9034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:01 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126921, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:01.869060 9034 init.cc:377] Logged in from keytab as oryx/127.29.156.195@KRBTEST.COM (short username oryx)
I20260807 18:22:01.870733 9034 webserver.cc:460] Webserver started at http://127.29.156.195:41501/ using document root <none> and password file <none>
I20260807 18:22:01.871477 9034 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:01.871660 9034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:01.871994 9034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:01.875223 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "312b1c66ca27488db65b9f5a775ed051"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "f782311a5205114a3cf1ebc51a086841"
server_key_iv: "c9e29df8d57298e65a4901a041331876"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.876156 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "312b1c66ca27488db65b9f5a775ed051"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "f782311a5205114a3cf1ebc51a086841"
server_key_iv: "c9e29df8d57298e65a4901a041331876"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.882126 9034 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:22:01.886241 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:01.888060 9034 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:22:01.888252 9034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "312b1c66ca27488db65b9f5a775ed051"
format_stamp: "Formatted at 2026-08-07 18:22:01 on dist-test-slave-sb2z"
server_key: "f782311a5205114a3cf1ebc51a086841"
server_key_iv: "c9e29df8d57298e65a4901a041331876"
server_key_version: "encryptionkey@0"
I20260807 18:22:01.888502 9034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:01.907902 9034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:01.913007 9034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:01.913335 9034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:01.916397 9034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:01.916527 9034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:01.916706 9034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:01.916795 9034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:22:01 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126921, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:01.942277 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.917213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:51133 (local address 127.29.156.254:36485)
0807 18:22:01.917903 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:01.917911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:01.918968 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 18:22:01.921456 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:01.921465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:01.921600 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:01.921775 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:01.924049 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.925043 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.928886 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.929531 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.929678 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.935141 (+ 5463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:01.935150 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:01.935155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:01.935200 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:01.938476 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.939257 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.939263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.939267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.939417 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.939935 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.939943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.939947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.940402 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:01.940584 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:01.941619 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0807 18:22:01.941859 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":426,"threads_started":1}
I20260807 18:22:01.943945 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.917865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.195:51133)
0807 18:22:01.918775 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:01.918826 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:01.921107 (+ 2281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:01.922122 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:01.922133 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:01.922953 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:01.923837 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.923860 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.925242 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.925250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.928718 (+ 3468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.928736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.929814 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.929825 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.930071 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.931034 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:01.931074 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:01.934854 (+ 3780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:01.938639 (+ 3785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.938648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.938667 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.939116 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.939571 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.939576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.939580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.939799 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.940565 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:01.940573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:01.941145 (+ 572us) client_negotiation.cc:769] Sending connection context
0807 18:22:01.941505 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 18:22:01.942132 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":412,"threads_started":1}
I20260807 18:22:01.969149 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.951903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:45121 (local address 127.29.156.254:36485)
0807 18:22:01.952109 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:01.952116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:01.953368 (+ 1252us) server_negotiation.cc:407] Connection header received
0807 18:22:01.953593 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:01.953597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:01.953666 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:01.953754 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:01.955598 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.956555 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.957972 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.958559 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.958732 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.961250 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:01.961255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:01.961258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:01.961292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:01.965744 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.966834 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.966843 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.966849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.967030 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:01.967415 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:01.967423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:01.967426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:01.967920 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:01.968101 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:01.968532 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:22:01.968785 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:22:01.969431 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:01.952142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.29.156.195:45121)
0807 18:22:01.953215 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:01.953250 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:01.953453 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:01.953847 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:01.953856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:01.954717 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:01.955277 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.955297 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.956711 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.956722 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.957821 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:01.957834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:01.958819 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:01.958826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:01.959154 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:01.960105 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:01.960125 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:01.960960 (+ 835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:01.965916 (+ 4956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.965923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.965927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.966593 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.967145 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:01.967150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:01.967155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:01.967293 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:01.968073 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:01.968078 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:01.968270 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 18:22:01.968430 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:22:01.968834 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":641,"threads_started":1}
I20260807 18:22:01.970698 9034 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:39023
I20260807 18:22:01.970728 9230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:39023 every 8 connection(s)
I20260807 18:22:01.972271 9034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:01.975085 9231 heartbeater.cc:347] Connected to a master server at 127.29.156.254:36485
I20260807 18:22:01.975299 9231 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:01.976110 9231 heartbeater.cc:511] Master 127.29.156.254:36485 requested a full tablet report, sending...
I20260807 18:22:01.977677 8545 ts_manager.cc:194] Registered new tserver with Master: 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023)
I20260807 18:22:01.978602 8545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:51133
I20260807 18:22:01.978883 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 9034
I20260807 18:22:01.979077 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:22:01.979755 30323 external_mini_cluster.cc:1428] Setting key dda81b30782f3b6016dbc1ef3022426b
I20260807 18:22:01.982921 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:22:02.057545 9072 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:02.041535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.29.156.254:36485)
0807 18:22:02.041782 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.041789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.042037 (+ 248us) server_negotiation.cc:407] Connection header received
0807 18:22:02.042227 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.042231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.042307 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.042396 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:02.044042 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.044804 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.046185 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.046623 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.046731 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.057227 (+ 10496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49610: BlockingRecv error: recv got EOF from 127.0.0.1:49610 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:02.075412 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.061270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.29.156.254:36485)
0807 18:22:02.061461 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.061465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.061813 (+ 348us) server_negotiation.cc:407] Connection header received
0807 18:22:02.061995 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.062000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.062088 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.062195 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:02.063446 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.064271 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.065495 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.065928 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.066025 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.069950 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:02.069955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:02.069958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:02.069992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:02.072872 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.073602 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.073606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.073608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.073726 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.074206 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.074210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.074212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.074482 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:02.074711 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.074986 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.075177 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:22:02.080112 8545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:22:02.087895 8545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:02.114609 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.102107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41485 (local address 127.0.0.1:37388)
0807 18:22:02.104209 (+ 2102us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.104276 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.104512 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.106384 (+ 1872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.106393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.106421 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.106912 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.106930 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.109248 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.109283 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.111658 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.111672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.113514 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.113521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.113715 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.113786 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.113915 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.114123 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1669,"mutex_wait_us":60,"thread_start_us":3745,"threads_started":1}
I20260807 18:22:02.115598 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.103772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.29.156.193:45739)
0807 18:22:02.105656 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.105667 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.106294 (+ 627us) server_negotiation.cc:407] Connection header received
0807 18:22:02.106644 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.106652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.106910 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.107095 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.108198 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.109472 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.112291 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.113994 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.114187 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.114366 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.115050 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.115238 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1704,"thread_start_us":1552,"threads_started":1}
I20260807 18:22:02.115583 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.102331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.29.156.194:41485)
0807 18:22:02.105203 (+ 2872us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.105215 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.105252 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:22:02.105751 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.105776 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.106098 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.107174 (+ 1076us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.107387 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.109069 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.111826 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.113244 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.113485 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.113680 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.115062 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.115242 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2704,"thread_start_us":2542,"threads_started":1}
I20260807 18:22:02.116145 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.103027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:45739 (local address 127.0.0.1:46776)
0807 18:22:02.106178 (+ 3151us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.106200 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.106405 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.107385 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.107390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.107417 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.107994 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.108010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.109666 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.109671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.112120 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.112133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.114584 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.114591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.114806 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.114839 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.115600 (+ 761us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.115738 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3004,"mutex_wait_us":32,"thread_start_us":958,"threads_started":1}
I20260807 18:22:02.116703 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.107427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:39023 (local address 127.0.0.1:47712)
0807 18:22:02.108056 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.108071 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.108301 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.108953 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.108958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.108984 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.109670 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.109683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.111842 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.111846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.113618 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.113632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.115883 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.115889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.116063 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.116123 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.116230 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.116389 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":355,"threads_started":1}
I20260807 18:22:02.116914 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.106685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.29.156.195:39023)
0807 18:22:02.107448 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.107456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.108187 (+ 731us) server_negotiation.cc:407] Connection header received
0807 18:22:02.108509 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.108518 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.108752 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.108936 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.109950 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.111666 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.113808 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.115242 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.115369 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.115467 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.116303 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.116492 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":421,"threads_started":1}
I20260807 18:22:02.120097 8923 tablet_service.cc:1467] Processing CreateTablet for tablet 67faf73073d8458f98ffd815dcf1178b (DEFAULT_TABLE table=test-table [id=c32347210c394a4fb5869a8ffca846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.120090 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 67faf73073d8458f98ffd815dcf1178b (DEFAULT_TABLE table=test-table [id=c32347210c394a4fb5869a8ffca846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.120424 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 67faf73073d8458f98ffd815dcf1178b (DEFAULT_TABLE table=test-table [id=c32347210c394a4fb5869a8ffca846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.121641 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67faf73073d8458f98ffd815dcf1178b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.121655 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67faf73073d8458f98ffd815dcf1178b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.121912 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67faf73073d8458f98ffd815dcf1178b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.135457 9261 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Bootstrap starting.
I20260807 18:22:02.138000 9264 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Bootstrap starting.
I20260807 18:22:02.138897 9262 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Bootstrap starting.
I20260807 18:22:02.139830 9261 tablet_bootstrap.cc:654] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.141378 9261 log.cc:824] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:02.143812 9264 tablet_bootstrap.cc:654] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.144670 9262 tablet_bootstrap.cc:654] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.145579 9264 log.cc:824] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:02.145891 9262 log.cc:824] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:02.149255 9261 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: No bootstrap required, opened a new log
I20260807 18:22:02.149478 9262 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: No bootstrap required, opened a new log
I20260807 18:22:02.149478 9264 tablet_bootstrap.cc:492] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: No bootstrap required, opened a new log
I20260807 18:22:02.149573 9261 ts_tablet_manager.cc:1397] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260807 18:22:02.149791 9262 ts_tablet_manager.cc:1397] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260807 18:22:02.149832 9264 ts_tablet_manager.cc:1397] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260807 18:22:02.158485 9261 raft_consensus.cc:348] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.158996 9261 raft_consensus.cc:374] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.159143 9261 raft_consensus.cc:729] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.159909 9261 consensus_queue.cc:260] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.161954 9231 heartbeater.cc:503] Master 127.29.156.254:36485 was elected leader, sending a full tablet report...
I20260807 18:22:02.162067 9264 raft_consensus.cc:348] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.162070 9262 raft_consensus.cc:348] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.162598 9264 raft_consensus.cc:374] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.162591 9262 raft_consensus.cc:374] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.162717 9262 raft_consensus.cc:729] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.162721 9264 raft_consensus.cc:729] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a59f504de14d1699431f305215cada, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.163430 9264 consensus_queue.cc:260] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.163430 9262 consensus_queue.cc:260] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.164520 9261 ts_tablet_manager.cc:1428] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260807 18:22:02.165671 9028 heartbeater.cc:503] Master 127.29.156.254:36485 was elected leader, sending a full tablet report...
I20260807 18:22:02.166566 9262 ts_tablet_manager.cc:1428] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Time spent starting tablet: real 0.017s user 0.018s sys 0.001s
I20260807 18:22:02.168395 9264 ts_tablet_manager.cc:1428] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Time spent starting tablet: real 0.018s user 0.015s sys 0.002s
I20260807 18:22:02.176803 9270 raft_consensus.cc:482] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:02.177070 9270 raft_consensus.cc:504] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.178862 9270 leader_election.cc:288] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485), c0a59f504de14d1699431f305215cada (127.29.156.193:45739)
I20260807 18:22:02.185757 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.179213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41485 (local address 127.29.156.195:52903)
0807 18:22:02.179442 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.179465 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.179631 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.180010 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.180015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.180042 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.180280 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.180290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.181622 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.181625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.183427 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.183442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.184955 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.184963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.185169 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.185206 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.185317 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.185457 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:22:02.185737 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.179265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:52903 (local address 127.29.156.194:41485)
0807 18:22:02.179486 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.179495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.179551 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:22:02.179806 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.179811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.179869 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.179978 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.180431 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.181471 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.183644 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.184753 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.184920 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.184999 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.185346 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.185475 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:22:02.186602 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.179627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:42843 (local address 127.29.156.193:45739)
0807 18:22:02.179882 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.179892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.180466 (+ 574us) server_negotiation.cc:407] Connection header received
0807 18:22:02.180629 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.180634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.180711 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.180811 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.181305 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.182107 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.184349 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.185536 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.185725 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.185798 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.186329 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.186430 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:22:02.186825 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.179303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:45739 (local address 127.29.156.195:42843)
0807 18:22:02.180380 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.180396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.180501 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.180831 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.180834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.180853 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.181143 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.181150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.182260 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.182266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.184156 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.184169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.185766 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.185773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.186139 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.186173 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.186294 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.186441 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":727,"threads_started":1}
I20260807 18:22:02.186872 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67faf73073d8458f98ffd815dcf1178b" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93" is_pre_election: true
I20260807 18:22:02.187496 8952 raft_consensus.cc:2454] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:02.188112 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67faf73073d8458f98ffd815dcf1178b" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada" is_pre_election: true
I20260807 18:22:02.188508 9059 leader_election.cc:302] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:02.188621 8770 raft_consensus.cc:2454] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:02.188966 9270 raft_consensus.cc:2781] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:02.189169 9270 raft_consensus.cc:482] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:02.189292 9270 raft_consensus.cc:3037] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.191785 9270 raft_consensus.cc:504] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.192466 9270 leader_election.cc:288] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Requested vote from peers 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485), c0a59f504de14d1699431f305215cada (127.29.156.193:45739)
I20260807 18:22:02.192926 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67faf73073d8458f98ffd815dcf1178b" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93"
I20260807 18:22:02.192989 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67faf73073d8458f98ffd815dcf1178b" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada"
I20260807 18:22:02.193163 8952 raft_consensus.cc:3037] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.193243 8770 raft_consensus.cc:3037] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.195600 8952 raft_consensus.cc:2454] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:02.196015 8770 raft_consensus.cc:2454] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:02.196053 9059 leader_election.cc:302] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:02.196352 9270 raft_consensus.cc:2781] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:02.196862 9270 raft_consensus.cc:686] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Becoming Leader. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Running, Role: LEADER
I20260807 18:22:02.197356 9270 consensus_queue.cc:237] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } }
I20260807 18:22:02.202267 8545 catalog_manager.cc:5495] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "312b1c66ca27488db65b9f5a775ed051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } health_report { overall_health: UNKNOWN } } }
W20260807 18:22:02.224418 9232 tablet.cc:2369] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:22:02.243718 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.237318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.29.156.195:39023)
0807 18:22:02.237601 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.237609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.237699 (+ 90us) server_negotiation.cc:407] Connection header received
0807 18:22:02.238001 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.238012 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.238108 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.238295 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:02.238720 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.239632 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.241131 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.241616 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.241780 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.242359 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.242984 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.243179 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.243247 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.243344 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:22:02.253969 8770 raft_consensus.cc:1261] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.253973 8952 raft_consensus.cc:1261] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.255357 9275 consensus_queue.cc:1035] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.255960 9270 consensus_queue.cc:1035] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.270988 9279 mvcc.cc:205] Tried to move back new op lower bound from 7315975873540038656 to 7315975873324359680. Current Snapshot: MvccSnapshot[applied={T|T < 7315975873540038656}]
I20260807 18:22:02.276757 9282 mvcc.cc:205] Tried to move back new op lower bound from 7315975873540038656 to 7315975873324359680. Current Snapshot: MvccSnapshot[applied={T|T < 7315975873540038656}]
I20260807 18:22:02.286012 8543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:02.286378 8543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:02.290896 8543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Sending DeleteTablet for 3 replicas of tablet 67faf73073d8458f98ffd815dcf1178b
I20260807 18:22:02.292114 8733 tablet_service.cc:1514] Processing DeleteTablet for tablet 67faf73073d8458f98ffd815dcf1178b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46776
I20260807 18:22:02.292160 8923 tablet_service.cc:1514] Processing DeleteTablet for tablet 67faf73073d8458f98ffd815dcf1178b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:37388
I20260807 18:22:02.292990 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 67faf73073d8458f98ffd815dcf1178b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:47712
I20260807 18:22:02.294919 8543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:22:02.296111 8543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:02.298162 9299 tablet_replica.cc:330] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: stopping tablet replica
I20260807 18:22:02.298679 9299 raft_consensus.cc:2229] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:22:02.299573 9299 raft_consensus.cc:2258] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.300362 9297 tablet_replica.cc:330] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: stopping tablet replica
I20260807 18:22:02.301311 9297 raft_consensus.cc:2229] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:02.301999 9297 raft_consensus.cc:2258] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.302934 9296 tablet_replica.cc:330] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: stopping tablet replica
I20260807 18:22:02.303583 9296 raft_consensus.cc:2229] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:02.304219 9296 raft_consensus.cc:2258] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.304353 9297 ts_tablet_manager.cc:1905] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.306378 9296 ts_tablet_manager.cc:1905] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.306437 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 (DEFAULT_TABLE table=test-table [id=7166c914431347a3ab73aee8ebef2975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.306943 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb6d84b069d46a59f6ff1327368a961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.311769 9297 ts_tablet_manager.cc:1918] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:02.311939 9297 log.cc:1196] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/67faf73073d8458f98ffd815dcf1178b
I20260807 18:22:02.311980 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 (DEFAULT_TABLE table=test-table [id=7166c914431347a3ab73aee8ebef2975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.312543 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb6d84b069d46a59f6ff1327368a961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.312566 9297 ts_tablet_manager.cc:1939] T 67faf73073d8458f98ffd815dcf1178b P 2cd4c058ecc9478c928c4edb6733ca93: Deleting consensus metadata
I20260807 18:22:02.314010 9296 ts_tablet_manager.cc:1918] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:02.314225 9296 log.cc:1196] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/67faf73073d8458f98ffd815dcf1178b
I20260807 18:22:02.314520 8520 catalog_manager.cc:4841] TS 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485): tablet 67faf73073d8458f98ffd815dcf1178b (table test-table [id=c32347210c394a4fb5869a8ffca846c9]) successfully deleted
I20260807 18:22:02.314894 9296 ts_tablet_manager.cc:1939] T 67faf73073d8458f98ffd815dcf1178b P c0a59f504de14d1699431f305215cada: Deleting consensus metadata
I20260807 18:22:02.316759 8521 catalog_manager.cc:4841] TS c0a59f504de14d1699431f305215cada (127.29.156.193:45739): tablet 67faf73073d8458f98ffd815dcf1178b (table test-table [id=c32347210c394a4fb5869a8ffca846c9]) successfully deleted
I20260807 18:22:02.317052 9262 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Bootstrap starting.
I20260807 18:22:02.318779 9262 tablet_bootstrap.cc:654] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.320305 9261 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Bootstrap starting.
I20260807 18:22:02.322125 9299 ts_tablet_manager.cc:1905] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.322220 9261 tablet_bootstrap.cc:654] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.321151 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 (DEFAULT_TABLE table=test-table [id=7166c914431347a3ab73aee8ebef2975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.324203 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb6d84b069d46a59f6ff1327368a961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.341547 9261 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: No bootstrap required, opened a new log
I20260807 18:22:02.341800 9261 ts_tablet_manager.cc:1397] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.014s
I20260807 18:22:02.342760 9261 raft_consensus.cc:348] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.343154 9261 raft_consensus.cc:374] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.343326 9261 raft_consensus.cc:729] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.343737 9261 consensus_queue.cc:260] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.347337 9261 ts_tablet_manager.cc:1428] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260807 18:22:02.347384 9262 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: No bootstrap required, opened a new log
I20260807 18:22:02.347645 9262 ts_tablet_manager.cc:1397] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent bootstrapping tablet: real 0.031s user 0.009s sys 0.021s
I20260807 18:22:02.348549 9262 raft_consensus.cc:348] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.348878 9262 raft_consensus.cc:374] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.349010 9262 raft_consensus.cc:729] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.349097 9299 ts_tablet_manager.cc:1918] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:02.349304 9299 log.cc:1196] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/67faf73073d8458f98ffd815dcf1178b
I20260807 18:22:02.349375 9262 consensus_queue.cc:260] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.350021 9299 ts_tablet_manager.cc:1939] T 67faf73073d8458f98ffd815dcf1178b P 312b1c66ca27488db65b9f5a775ed051: Deleting consensus metadata
I20260807 18:22:02.350363 9262 ts_tablet_manager.cc:1428] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:22:02.351783 8520 catalog_manager.cc:4841] TS 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023): tablet 67faf73073d8458f98ffd815dcf1178b (table test-table [id=c32347210c394a4fb5869a8ffca846c9]) successfully deleted
I20260807 18:22:02.352106 9264 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Bootstrap starting.
I20260807 18:22:02.353941 9264 tablet_bootstrap.cc:654] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.356235 9264 tablet_bootstrap.cc:492] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: No bootstrap required, opened a new log
I20260807 18:22:02.356395 9264 ts_tablet_manager.cc:1397] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:22:02.357200 9264 raft_consensus.cc:348] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.357570 9264 raft_consensus.cc:374] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.357694 9264 raft_consensus.cc:729] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a59f504de14d1699431f305215cada, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.358016 9264 consensus_queue.cc:260] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.358155 9275 raft_consensus.cc:482] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:02.358407 9275 raft_consensus.cc:504] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.358721 9264 ts_tablet_manager.cc:1428] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 18:22:02.359155 9275 leader_election.cc:288] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485)
I20260807 18:22:02.359777 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb6d84b069d46a59f6ff1327368a961" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada" is_pre_election: true
I20260807 18:22:02.360034 8770 raft_consensus.cc:2454] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:02.360118 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb6d84b069d46a59f6ff1327368a961" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93" is_pre_election: true
I20260807 18:22:02.360374 8952 raft_consensus.cc:2454] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:02.360970 9059 leader_election.cc:302] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:02.362154 9275 raft_consensus.cc:2781] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:02.362282 9275 raft_consensus.cc:482] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:02.362411 9275 raft_consensus.cc:3037] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.365056 9275 raft_consensus.cc:504] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.365769 9275 leader_election.cc:288] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Requested vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485)
I20260807 18:22:02.366117 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb6d84b069d46a59f6ff1327368a961" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada"
I20260807 18:22:02.366318 8770 raft_consensus.cc:3037] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.366331 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb6d84b069d46a59f6ff1327368a961" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93"
I20260807 18:22:02.366572 8952 raft_consensus.cc:3037] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.368880 8952 raft_consensus.cc:2454] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:02.369278 8770 raft_consensus.cc:2454] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:02.369354 9059 leader_election.cc:302] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:02.369699 9275 raft_consensus.cc:2781] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:02.370204 9275 raft_consensus.cc:686] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Becoming Leader. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Running, Role: LEADER
I20260807 18:22:02.370494 9275 consensus_queue.cc:237] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.373711 8543 catalog_manager.cc:5495] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "312b1c66ca27488db65b9f5a775ed051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } health_report { overall_health: HEALTHY } } }
W20260807 18:22:02.397416 9030 tablet.cc:2369] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126919, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.254@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:02.435101 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.418399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.29.156.254:36485)
0807 18:22:02.418651 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.418657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.419306 (+ 649us) server_negotiation.cc:407] Connection header received
0807 18:22:02.419482 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.419489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.419596 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.419705 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:02.421276 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.422470 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.424290 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.424943 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.425074 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.428970 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:02.428977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:02.428981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:02.429021 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:02.431981 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.432977 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.432981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.432983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.433125 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.433534 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.433541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.433544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.433809 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:02.434000 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.434423 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.434666 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:22:02.435220 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.418428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36485 (local address 127.0.0.1:49630)
0807 18:22:02.419185 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.419206 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.419345 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.419766 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.419770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.420710 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:02.421113 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.421125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.422656 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.422664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.424143 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.424157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.425136 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.425141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.425275 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.426028 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:02.426046 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:02.428775 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:02.432105 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:02.432119 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:02.432127 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:02.432735 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:02.433262 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:02.433267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:02.433272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:02.433420 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:02.433985 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:02.433994 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:02.434167 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.434344 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.434801 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":163,"thread_start_us":418,"threads_started":1}
I20260807 18:22:02.441000 8543 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:49630:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:22:02.442137 8543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:02.450445 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 984a85dcf20b466ba7d171a55a4eb68e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f4fbdb7f0a54598b62973ae02ebace6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:22:02.451056 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984a85dcf20b466ba7d171a55a4eb68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.452155 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 984a85dcf20b466ba7d171a55a4eb68e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f4fbdb7f0a54598b62973ae02ebace6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:22:02.452683 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984a85dcf20b466ba7d171a55a4eb68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.458052 9264 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: Bootstrap starting.
I20260807 18:22:02.458407 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 984a85dcf20b466ba7d171a55a4eb68e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f4fbdb7f0a54598b62973ae02ebace6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:22:02.459028 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984a85dcf20b466ba7d171a55a4eb68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.459816 9264 tablet_bootstrap.cc:654] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.462100 9264 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: No bootstrap required, opened a new log
I20260807 18:22:02.462311 9264 ts_tablet_manager.cc:1397] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:22:02.462944 9264 raft_consensus.cc:348] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.463197 9264 raft_consensus.cc:374] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.463316 9264 raft_consensus.cc:729] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a59f504de14d1699431f305215cada, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.463658 9264 consensus_queue.cc:260] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.464183 9271 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.464577 9271 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.465400 9264 ts_tablet_manager.cc:1428] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:02.466696 9261 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: Bootstrap starting.
I20260807 18:22:02.468498 9261 tablet_bootstrap.cc:654] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.469632 9262 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: Bootstrap starting.
I20260807 18:22:02.471217 9262 tablet_bootstrap.cc:654] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.473914 9262 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: No bootstrap required, opened a new log
I20260807 18:22:02.473960 9261 tablet_bootstrap.cc:492] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: No bootstrap required, opened a new log
I20260807 18:22:02.474089 9262 ts_tablet_manager.cc:1397] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:22:02.474263 9261 ts_tablet_manager.cc:1397] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 18:22:02.474865 9262 raft_consensus.cc:348] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.474979 9261 raft_consensus.cc:348] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.475126 9262 raft_consensus.cc:374] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.475229 9262 raft_consensus.cc:729] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.475261 9261 raft_consensus.cc:374] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.475387 9261 raft_consensus.cc:729] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.475471 9262 consensus_queue.cc:260] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.475718 9261 consensus_queue.cc:260] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.476128 9262 ts_tablet_manager.cc:1428] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 18:22:02.476408 9261 ts_tablet_manager.cc:1428] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:22:02.476326 9275 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.476624 9275 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.476964 9273 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.477372 9273 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.486038 9273 raft_consensus.cc:482] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:02.486263 9273 raft_consensus.cc:504] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.487906 9273 leader_election.cc:288] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023)
I20260807 18:22:02.497323 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.488925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35939 (local address 127.29.156.193:45739)
0807 18:22:02.489110 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.489116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.489136 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:22:02.489351 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.489358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.489461 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.489541 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.490718 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.491913 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.494894 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.496202 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.496383 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.496466 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.496868 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.497012 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:22:02.497800 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.488408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:59923 (local address 127.29.156.195:39023)
0807 18:22:02.488694 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.488701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.489703 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 18:22:02.489881 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.489887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.489980 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.490095 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:02.491342 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.492609 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.495346 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.496414 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.496604 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.496689 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.497303 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.497489 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260807 18:22:02.497836 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.488174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:45739 (local address 127.29.156.194:35939)
0807 18:22:02.488964 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.488994 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.489133 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.489764 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.489770 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.489819 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.490494 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.490516 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.492037 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.492046 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.494699 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.494717 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.496477 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.496489 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.496676 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.496718 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.496848 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.497012 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":2,"thread_start_us":798,"threads_started":1}
I20260807 18:22:02.498557 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.488322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:39023 (local address 127.29.156.194:59923)
0807 18:22:02.489583 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.489601 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.489735 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.490123 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.490128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.490154 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:02.490594 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.490607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.492750 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.492755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.495161 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.495173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.496872 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.496878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.497078 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.497117 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.498036 (+ 919us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.498182 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":926,"threads_started":1}
I20260807 18:22:02.498783 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984a85dcf20b466ba7d171a55a4eb68e" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada" is_pre_election: true
I20260807 18:22:02.499070 8770 raft_consensus.cc:2454] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 0.
I20260807 18:22:02.499663 8855 leader_election.cc:302] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, c0a59f504de14d1699431f305215cada; no voters:
I20260807 18:22:02.499615 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984a85dcf20b466ba7d171a55a4eb68e" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312b1c66ca27488db65b9f5a775ed051" is_pre_election: true
I20260807 18:22:02.500129 9273 raft_consensus.cc:2781] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:02.500077 9161 raft_consensus.cc:2454] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 0.
I20260807 18:22:02.500283 9273 raft_consensus.cc:482] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:02.500391 9273 raft_consensus.cc:3037] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.503096 9273 raft_consensus.cc:504] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.503942 9273 leader_election.cc:288] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 election: Requested vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023)
I20260807 18:22:02.504384 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984a85dcf20b466ba7d171a55a4eb68e" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada"
I20260807 18:22:02.504593 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984a85dcf20b466ba7d171a55a4eb68e" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312b1c66ca27488db65b9f5a775ed051"
I20260807 18:22:02.504699 8770 raft_consensus.cc:3037] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.504825 9161 raft_consensus.cc:3037] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:02.506994 8770 raft_consensus.cc:2454] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 1.
I20260807 18:22:02.507417 8855 leader_election.cc:302] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, c0a59f504de14d1699431f305215cada; no voters:
I20260807 18:22:02.507829 9273 raft_consensus.cc:2781] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:02.508363 9161 raft_consensus.cc:2454] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 1.
I20260807 18:22:02.508426 9273 raft_consensus.cc:686] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Running, Role: LEADER
I20260807 18:22:02.508934 9273 consensus_queue.cc:237] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.512086 9318 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: TxnStatusTablet state changed. Reason: New leader 2cd4c058ecc9478c928c4edb6733ca93. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.512547 9318 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:22:02.513348 9321 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:22:02.513589 9321 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:22:02.514232 8545 catalog_manager.cc:5495] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } health_report { overall_health: UNKNOWN } } }
I20260807 18:22:02.542796 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.534604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41485 (local address 127.0.0.1:37398)
0807 18:22:02.534984 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.535007 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.535170 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.535806 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.535811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.535824 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:22:02.536030 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.536041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.537764 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.537770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.539360 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.539376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.540587 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.540593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.540834 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.540927 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.542155 (+ 1228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.542187 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.542284 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.542452 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260807 18:22:02.542829 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.534572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37398 (local address 127.29.156.194:41485)
0807 18:22:02.534788 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.534797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.535104 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:22:02.535403 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.535411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.535579 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.535768 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:02.536201 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.537613 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.539584 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.540385 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.540545 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.541164 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.542017 (+ 853us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.542193 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.542334 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.542440 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:22:02.548172 9161 raft_consensus.cc:1261] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Refusing update from remote peer 2cd4c058ecc9478c928c4edb6733ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.548254 8770 raft_consensus.cc:1261] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Refusing update from remote peer 2cd4c058ecc9478c928c4edb6733ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.549083 9273 consensus_queue.cc:1035] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.550125 9273 consensus_queue.cc:1035] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.559099 9271 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: TxnStatusTablet state changed. Reason: New leader 2cd4c058ecc9478c928c4edb6733ca93. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.559504 9271 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.563660 9273 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.564010 9273 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:22:02.563838 9275 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: TxnStatusTablet state changed. Reason: New leader 2cd4c058ecc9478c928c4edb6733ca93. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.564185 9275 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.568149 9281 mvcc.cc:205] Tried to move back new op lower bound from 7315975874748014592 to 7315975874601000960. Current Snapshot: MvccSnapshot[applied={T|T < 7315975874748014592 or (T in {7315975874748014592})}]
I20260807 18:22:02.569869 9275 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.570200 9275 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 312b1c66ca27488db65b9f5a775ed051: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.576726 9318 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.577059 9318 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P 2cd4c058ecc9478c928c4edb6733ca93: This TxnStatusTablet replica's current role is: LEADER
W20260807 18:22:02.578896 8830 tablet.cc:2369] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:22:02.580128 9271 tablet_replica.cc:439] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } } }
I20260807 18:22:02.580401 9271 tablet_replica.cc:442] T 984a85dcf20b466ba7d171a55a4eb68e P c0a59f504de14d1699431f305215cada: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:22:02.600417 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.591936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:48501 (local address 127.29.156.194:41485)
0807 18:22:02.592207 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.592214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.593003 (+ 789us) server_negotiation.cc:407] Connection header received
0807 18:22:02.593261 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.593267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.593374 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.593486 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:02.594250 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.595520 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.597877 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.598653 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.598800 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.599423 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.599619 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.599728 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.599904 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.600067 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:22:02.600476 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.592051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41485 (local address 127.29.156.195:48501)
0807 18:22:02.592877 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.592897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.593060 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.593537 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.593542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.593602 (+ 60us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:22:02.594033 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.594047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.595713 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.595719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.597691 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.597714 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.598887 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.598895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.599107 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.599192 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.599744 (+ 552us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.599758 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.599864 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.600020 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":425,"threads_started":1}
I20260807 18:22:02.613411 8770 raft_consensus.cc:1261] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.613435 8952 raft_consensus.cc:1261] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:02.614046 9290 consensus_queue.cc:1035] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.614670 9275 consensus_queue.cc:1035] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:02.670004 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.657375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35561 (local address 127.29.156.195:39023)
0807 18:22:02.657648 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.657657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.662311 (+ 4654us) server_negotiation.cc:407] Connection header received
0807 18:22:02.662612 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.662618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.662705 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.662765 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:02.663653 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.664881 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.667464 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.668262 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.668407 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.669105 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.669277 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.669370 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.669585 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.669679 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:22:02.671509 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.656849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:39023 (local address 127.29.156.194:35561)
0807 18:22:02.662204 (+ 5355us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:02.662225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:02.662435 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:02.662975 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:02.662980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:02.662993 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:22:02.663402 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.663414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.665138 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.665145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.667284 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:02.667302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.668519 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.668533 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:02.668762 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.668863 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:22:02.669425 (+ 562us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:22:02.669440 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:22:02.670290 (+ 850us) client_negotiation.cc:240] Negotiation successful
0807 18:22:02.670461 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5094,"thread_start_us":4918,"threads_started":1}
I20260807 18:22:02.724680 8545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:02.725021 8545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:02.729182 8545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Sending DeleteTablet for 3 replicas of tablet 8fb6d84b069d46a59f6ff1327368a961
I20260807 18:22:02.730360 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:47712
I20260807 18:22:02.730468 8733 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46776
I20260807 18:22:02.730690 9299 tablet_replica.cc:330] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: stopping tablet replica
I20260807 18:22:02.730860 9296 tablet_replica.cc:330] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: stopping tablet replica
I20260807 18:22:02.730906 9299 raft_consensus.cc:2229] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:22:02.731091 9296 raft_consensus.cc:2229] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:02.731282 9296 raft_consensus.cc:2258] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.731299 9299 raft_consensus.cc:2258] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.732779 9296 ts_tablet_manager.cc:1905] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.732779 9299 ts_tablet_manager.cc:1905] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.741132 9299 ts_tablet_manager.cc:1918] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:02.741132 9296 ts_tablet_manager.cc:1918] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:02.741349 9296 log.cc:1196] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/8fb6d84b069d46a59f6ff1327368a961
I20260807 18:22:02.741351 9299 log.cc:1196] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/8fb6d84b069d46a59f6ff1327368a961
I20260807 18:22:02.741925 9299 ts_tablet_manager.cc:1939] T 8fb6d84b069d46a59f6ff1327368a961 P 312b1c66ca27488db65b9f5a775ed051: Deleting consensus metadata
I20260807 18:22:02.741900 9296 ts_tablet_manager.cc:1939] T 8fb6d84b069d46a59f6ff1327368a961 P c0a59f504de14d1699431f305215cada: Deleting consensus metadata
I20260807 18:22:02.741978 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb6d84b069d46a59f6ff1327368a961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:02 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:37388
I20260807 18:22:02.742357 9297 tablet_replica.cc:330] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: stopping tablet replica
I20260807 18:22:02.742626 9297 raft_consensus.cc:2229] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:02.742882 9297 raft_consensus.cc:2258] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:02.743342 8521 catalog_manager.cc:4841] TS c0a59f504de14d1699431f305215cada (127.29.156.193:45739): tablet 8fb6d84b069d46a59f6ff1327368a961 (table test-table [id=7166c914431347a3ab73aee8ebef2975]) successfully deleted
I20260807 18:22:02.744529 9297 ts_tablet_manager.cc:1905] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:02.752756 9297 ts_tablet_manager.cc:1918] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:02.752945 9297 log.cc:1196] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/8fb6d84b069d46a59f6ff1327368a961
I20260807 18:22:02.752883 8520 catalog_manager.cc:4841] TS 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023): tablet 8fb6d84b069d46a59f6ff1327368a961 (table test-table [id=7166c914431347a3ab73aee8ebef2975]) successfully deleted
I20260807 18:22:02.753448 9297 ts_tablet_manager.cc:1939] T 8fb6d84b069d46a59f6ff1327368a961 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting consensus metadata
I20260807 18:22:02.754662 8520 catalog_manager.cc:4841] TS 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485): tablet 8fb6d84b069d46a59f6ff1327368a961 (table test-table [id=7166c914431347a3ab73aee8ebef2975]) successfully deleted
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:22:02.834619 9072 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:02.817824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.29.156.254:36485)
0807 18:22:02.818080 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.818087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.818742 (+ 655us) server_negotiation.cc:407] Connection header received
0807 18:22:02.819011 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.819018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.819104 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.819211 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:02.821928 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.823307 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.825543 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.826301 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.826426 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.834319 (+ 7893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49632: BlockingRecv error: recv got EOF from 127.0.0.1:49632 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 18:22:02 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM
I20260807 18:22:02.898599 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:02.875975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.29.156.254:36485)
0807 18:22:02.876230 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:02.876237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:02.876996 (+ 759us) server_negotiation.cc:407] Connection header received
0807 18:22:02.877291 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:02.877298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:02.877413 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:02.877570 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:02.879812 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.881154 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.883300 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:02.884150 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:02.884300 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:02.890485 (+ 6185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:02.890494 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:02.890508 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:02.890565 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:02.894756 (+ 4191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.895825 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.895833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.895837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.896005 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:02.896547 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:02.896555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:02.896560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:02.896980 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:02.897258 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:02.897689 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:22:02.897956 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 18:22:02.903276 8545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:22:02.904086 8545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:02.913049 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 120db98f7232431f96aea64be6397b61 (DEFAULT_TABLE table=test-table [id=80b3388e4ba4462ca44be06ed401dc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.913543 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 120db98f7232431f96aea64be6397b61 (DEFAULT_TABLE table=test-table [id=80b3388e4ba4462ca44be06ed401dc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.913669 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120db98f7232431f96aea64be6397b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.914055 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120db98f7232431f96aea64be6397b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.918324 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 120db98f7232431f96aea64be6397b61 (DEFAULT_TABLE table=test-table [id=80b3388e4ba4462ca44be06ed401dc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:02.918821 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120db98f7232431f96aea64be6397b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:02.921952 9262 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Bootstrap starting.
I20260807 18:22:02.924183 9262 tablet_bootstrap.cc:654] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.925894 9261 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Bootstrap starting.
I20260807 18:22:02.926811 9264 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Bootstrap starting.
I20260807 18:22:02.928051 9261 tablet_bootstrap.cc:654] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.928678 9264 tablet_bootstrap.cc:654] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:02.928828 9262 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: No bootstrap required, opened a new log
I20260807 18:22:02.928989 9262 ts_tablet_manager.cc:1397] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:22:02.930003 9262 raft_consensus.cc:348] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.930420 9262 raft_consensus.cc:374] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.930554 9262 raft_consensus.cc:729] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.930871 9262 consensus_queue.cc:260] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.931689 9262 ts_tablet_manager.cc:1428] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:02.933266 9264 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: No bootstrap required, opened a new log
I20260807 18:22:02.933429 9264 ts_tablet_manager.cc:1397] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260807 18:22:02.934283 9264 raft_consensus.cc:348] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.934520 9264 raft_consensus.cc:374] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.934633 9264 raft_consensus.cc:729] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a59f504de14d1699431f305215cada, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.934892 9264 consensus_queue.cc:260] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.936094 9264 ts_tablet_manager.cc:1428] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 18:22:02.939785 9261 tablet_bootstrap.cc:492] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: No bootstrap required, opened a new log
I20260807 18:22:02.940001 9261 ts_tablet_manager.cc:1397] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.001s
I20260807 18:22:02.940675 9261 raft_consensus.cc:348] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.940922 9261 raft_consensus.cc:374] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:02.941045 9261 raft_consensus.cc:729] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Initialized, Role: FOLLOWER
I20260807 18:22:02.941329 9261 consensus_queue.cc:260] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:02.942030 9261 ts_tablet_manager.cc:1428] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:22:03.012933 9290 raft_consensus.cc:482] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:03.013175 9290 raft_consensus.cc:504] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.013871 9290 leader_election.cc:288] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485)
I20260807 18:22:03.014488 8951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120db98f7232431f96aea64be6397b61" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93" is_pre_election: true
I20260807 18:22:03.014526 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120db98f7232431f96aea64be6397b61" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada" is_pre_election: true
I20260807 18:22:03.014798 8770 raft_consensus.cc:2454] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:03.014803 8951 raft_consensus.cc:2454] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 0.
I20260807 18:22:03.015309 9059 leader_election.cc:302] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:03.015658 9290 raft_consensus.cc:2781] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:03.015758 9290 raft_consensus.cc:482] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:03.015879 9290 raft_consensus.cc:3037] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.018187 9290 raft_consensus.cc:504] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.018791 9290 leader_election.cc:288] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Requested vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485)
I20260807 18:22:03.019315 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120db98f7232431f96aea64be6397b61" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada"
I20260807 18:22:03.019392 8951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120db98f7232431f96aea64be6397b61" candidate_uuid: "312b1c66ca27488db65b9f5a775ed051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4c058ecc9478c928c4edb6733ca93"
I20260807 18:22:03.019564 8770 raft_consensus.cc:3037] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.019708 8951 raft_consensus.cc:3037] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.021931 8770 raft_consensus.cc:2454] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:03.022431 9061 leader_election.cc:302] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 312b1c66ca27488db65b9f5a775ed051, c0a59f504de14d1699431f305215cada; no voters:
I20260807 18:22:03.022811 9290 raft_consensus.cc:2781] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:03.023018 9290 raft_consensus.cc:686] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Becoming Leader. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Running, Role: LEADER
I20260807 18:22:03.023276 9290 consensus_queue.cc:237] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.023411 8951 raft_consensus.cc:2454] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312b1c66ca27488db65b9f5a775ed051 in term 1.
I20260807 18:22:03.026719 8546 catalog_manager.cc:5495] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "312b1c66ca27488db65b9f5a775ed051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } health_report { overall_health: HEALTHY } } }
I20260807 18:22:03.044802 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.038165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.29.156.195:39023)
0807 18:22:03.038355 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.038360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.038675 (+ 315us) server_negotiation.cc:407] Connection header received
0807 18:22:03.038926 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.038931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.039004 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.039101 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:03.039648 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.040380 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.042628 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.043114 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.043215 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.043990 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:03.044127 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:03.044215 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.044422 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.044495 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:22:03.047919 8951 raft_consensus.cc:1261] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:03.048254 8770 raft_consensus.cc:1261] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Refusing update from remote peer 312b1c66ca27488db65b9f5a775ed051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:03.048969 9290 consensus_queue.cc:1035] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:03.049330 9275 consensus_queue.cc:1035] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:03.064924 8543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:03.065171 8543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:03.070472 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 120db98f7232431f96aea64be6397b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:37388
I20260807 18:22:03.069633 8543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Sending DeleteTablet for 3 replicas of tablet 120db98f7232431f96aea64be6397b61
I20260807 18:22:03.070851 9297 tablet_replica.cc:330] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: stopping tablet replica
I20260807 18:22:03.071092 9297 raft_consensus.cc:2229] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:03.071367 9297 raft_consensus.cc:2258] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.072108 8733 tablet_service.cc:1514] Processing DeleteTablet for tablet 120db98f7232431f96aea64be6397b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46776
I20260807 18:22:03.072438 9296 tablet_replica.cc:330] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: stopping tablet replica
I20260807 18:22:03.072642 9296 raft_consensus.cc:2229] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:03.072736 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 120db98f7232431f96aea64be6397b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:47712
I20260807 18:22:03.072901 9296 raft_consensus.cc:2258] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.073208 9299 tablet_replica.cc:330] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: stopping tablet replica
I20260807 18:22:03.073400 9299 raft_consensus.cc:2229] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:22:03.073758 9299 raft_consensus.cc:2258] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.073982 8541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:22:03.074257 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 120db98f7232431f96aea64be6397b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:37388
I20260807 18:22:03.074596 9297 ts_tablet_manager.cc:1905] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:22:03.074697 8541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:03.075276 9299 ts_tablet_manager.cc:1905] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:03.080320 9296 ts_tablet_manager.cc:1905] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:03.084126 9297 ts_tablet_manager.cc:1918] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:03.087852 9299 ts_tablet_manager.cc:1918] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:03.088088 9299 log.cc:1196] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/120db98f7232431f96aea64be6397b61
I20260807 18:22:03.088009 9297 log.cc:1196] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/120db98f7232431f96aea64be6397b61
I20260807 18:22:03.088629 9296 ts_tablet_manager.cc:1918] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:03.088781 9296 log.cc:1196] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/120db98f7232431f96aea64be6397b61
I20260807 18:22:03.088886 9299 ts_tablet_manager.cc:1939] T 120db98f7232431f96aea64be6397b61 P 312b1c66ca27488db65b9f5a775ed051: Deleting consensus metadata
I20260807 18:22:03.089219 9296 ts_tablet_manager.cc:1939] T 120db98f7232431f96aea64be6397b61 P c0a59f504de14d1699431f305215cada: Deleting consensus metadata
I20260807 18:22:03.090432 8521 catalog_manager.cc:4841] TS c0a59f504de14d1699431f305215cada (127.29.156.193:45739): tablet 120db98f7232431f96aea64be6397b61 (table test-table [id=80b3388e4ba4462ca44be06ed401dc23]) successfully deleted
I20260807 18:22:03.090559 8520 catalog_manager.cc:4841] TS 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023): tablet 120db98f7232431f96aea64be6397b61 (table test-table [id=80b3388e4ba4462ca44be06ed401dc23]) successfully deleted
I20260807 18:22:03.090770 9297 ts_tablet_manager.cc:1939] T 120db98f7232431f96aea64be6397b61 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting consensus metadata
I20260807 18:22:03.092315 8520 catalog_manager.cc:4841] TS 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485): tablet 120db98f7232431f96aea64be6397b61 (table test-table [id=80b3388e4ba4462ca44be06ed401dc23]) successfully deleted
I20260807 18:22:03.093094 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (DEFAULT_TABLE table=test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260807 18:22:03.093467 8520 catalog_manager.cc:4808] TS 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485): delete failed for tablet 120db98f7232431f96aea64be6397b61 because the tablet was not found. No further retry: Not found: Tablet not found: 120db98f7232431f96aea64be6397b61
I20260807 18:22:03.093585 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:03.097280 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (DEFAULT_TABLE table=test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:03.097831 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:03.106108 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (DEFAULT_TABLE table=test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:03.106667 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:03.117273 9262 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Bootstrap starting.
I20260807 18:22:03.117677 9261 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Bootstrap starting.
I20260807 18:22:03.118624 9264 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Bootstrap starting.
I20260807 18:22:03.119201 9262 tablet_bootstrap.cc:654] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:03.119478 9261 tablet_bootstrap.cc:654] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:03.120601 9264 tablet_bootstrap.cc:654] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:03.122205 9262 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: No bootstrap required, opened a new log
I20260807 18:22:03.122454 9262 ts_tablet_manager.cc:1397] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260807 18:22:03.123606 9262 raft_consensus.cc:348] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.124010 9262 raft_consensus.cc:374] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:03.124125 9262 raft_consensus.cc:729] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Initialized, Role: FOLLOWER
I20260807 18:22:03.124470 9262 consensus_queue.cc:260] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.125159 9262 ts_tablet_manager.cc:1428] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 18:22:03.128265 9261 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: No bootstrap required, opened a new log
I20260807 18:22:03.128484 9261 ts_tablet_manager.cc:1397] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260807 18:22:03.129426 9261 raft_consensus.cc:348] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.130100 9261 raft_consensus.cc:374] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:03.130205 9261 raft_consensus.cc:729] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312b1c66ca27488db65b9f5a775ed051, State: Initialized, Role: FOLLOWER
I20260807 18:22:03.130510 9261 consensus_queue.cc:260] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.131130 9261 ts_tablet_manager.cc:1428] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 18:22:03.133145 9264 tablet_bootstrap.cc:492] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: No bootstrap required, opened a new log
I20260807 18:22:03.133352 9264 ts_tablet_manager.cc:1397] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260807 18:22:03.134258 9264 raft_consensus.cc:348] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.134559 9264 raft_consensus.cc:374] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:03.134661 9264 raft_consensus.cc:729] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a59f504de14d1699431f305215cada, State: Initialized, Role: FOLLOWER
I20260807 18:22:03.134955 9264 consensus_queue.cc:260] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.135687 9264 ts_tablet_manager.cc:1428] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 18:22:03.256057 9335 raft_consensus.cc:482] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:03.256258 9335 raft_consensus.cc:504] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.257069 9335 leader_election.cc:288] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023)
I20260807 18:22:03.257658 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efd22bb60fb4e1a889a4ee85e0d92b1" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada" is_pre_election: true
I20260807 18:22:03.257700 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efd22bb60fb4e1a889a4ee85e0d92b1" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312b1c66ca27488db65b9f5a775ed051" is_pre_election: true
I20260807 18:22:03.257930 8770 raft_consensus.cc:2454] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 0.
I20260807 18:22:03.257939 9161 raft_consensus.cc:2454] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 0.
I20260807 18:22:03.258421 8854 leader_election.cc:302] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, 312b1c66ca27488db65b9f5a775ed051; no voters:
I20260807 18:22:03.258749 9335 raft_consensus.cc:2781] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:03.258915 9335 raft_consensus.cc:482] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:03.259025 9335 raft_consensus.cc:3037] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.261253 9335 raft_consensus.cc:504] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.261912 9335 leader_election.cc:288] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 election: Requested vote from peers c0a59f504de14d1699431f305215cada (127.29.156.193:45739), 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023)
I20260807 18:22:03.262516 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efd22bb60fb4e1a889a4ee85e0d92b1" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a59f504de14d1699431f305215cada"
I20260807 18:22:03.262569 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efd22bb60fb4e1a889a4ee85e0d92b1" candidate_uuid: "2cd4c058ecc9478c928c4edb6733ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312b1c66ca27488db65b9f5a775ed051"
I20260807 18:22:03.262761 8770 raft_consensus.cc:3037] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.262826 9161 raft_consensus.cc:3037] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:03.264945 8770 raft_consensus.cc:2454] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 1.
I20260807 18:22:03.265300 9161 raft_consensus.cc:2454] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd4c058ecc9478c928c4edb6733ca93 in term 1.
I20260807 18:22:03.265445 8855 leader_election.cc:302] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4c058ecc9478c928c4edb6733ca93, c0a59f504de14d1699431f305215cada; no voters:
I20260807 18:22:03.265813 9335 raft_consensus.cc:2781] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:03.266216 9335 raft_consensus.cc:686] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd4c058ecc9478c928c4edb6733ca93, State: Running, Role: LEADER
I20260807 18:22:03.266507 9335 consensus_queue.cc:237] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } }
I20260807 18:22:03.269121 8541 catalog_manager.cc:5495] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "2cd4c058ecc9478c928c4edb6733ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd4c058ecc9478c928c4edb6733ca93" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 } health_report { overall_health: UNKNOWN } } }
I20260807 18:22:03.312420 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.306873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.29.156.194:41485)
0807 18:22:03.307705 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.307709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.307726 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:22:03.307782 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.307786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.307851 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.307941 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:03.308304 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.309161 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.310470 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.310931 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.311035 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.311490 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:03.311675 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:03.311753 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.311925 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.311991 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":511,"threads_started":1}
I20260807 18:22:03.336203 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.330313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:40659 (local address 127.29.156.194:41485)
0807 18:22:03.330543 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.330552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.330972 (+ 420us) server_negotiation.cc:407] Connection header received
0807 18:22:03.331202 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.331207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.331288 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.331393 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:03.331998 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.332693 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.334353 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.334755 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.334853 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.335397 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:03.335509 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:03.335662 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.335779 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.335845 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:22:03.336249 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.330240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41485 (local address 127.29.156.194:40659)
0807 18:22:03.330855 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:03.330877 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:03.331053 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:03.331446 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:03.331450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:03.331470 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:22:03.331816 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:03.331825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.332829 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.332833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:03.334190 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:03.334203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.334959 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.334971 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:03.335127 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.335170 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:22:03.335661 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:22:03.335672 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:22:03.335761 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:22:03.335898 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":306,"threads_started":1}
I20260807 18:22:03.345085 9161 raft_consensus.cc:1261] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Refusing update from remote peer 2cd4c058ecc9478c928c4edb6733ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:03.345221 8770 raft_consensus.cc:1261] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Refusing update from remote peer 2cd4c058ecc9478c928c4edb6733ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:03.345713 9335 consensus_queue.cc:1035] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "312b1c66ca27488db65b9f5a775ed051" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 39023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:03.345983 9379 consensus_queue.cc:1035] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a59f504de14d1699431f305215cada" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:03.395466 8543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:03.395808 8543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:03.398888 8543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a735ba83ef479a8ecf2c137edda320: Sending DeleteTablet for 3 replicas of tablet 6efd22bb60fb4e1a889a4ee85e0d92b1
I20260807 18:22:03.399783 8733 tablet_service.cc:1514] Processing DeleteTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46776
I20260807 18:22:03.400141 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:47712
I20260807 18:22:03.400471 9296 tablet_replica.cc:330] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: stopping tablet replica
I20260807 18:22:03.400703 9296 raft_consensus.cc:2229] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:03.400858 9296 raft_consensus.cc:2258] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.401153 9299 tablet_replica.cc:330] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: stopping tablet replica
I20260807 18:22:03.401366 9299 raft_consensus.cc:2229] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:03.401559 9299 raft_consensus.cc:2258] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.401957 9296 ts_tablet_manager.cc:1905] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:03.403595 9299 ts_tablet_manager.cc:1905] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:03.403571 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:03 UTC) from {username='oryx', principal='oryx/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:37388
I20260807 18:22:03.403915 9297 tablet_replica.cc:330] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: stopping tablet replica
I20260807 18:22:03.404143 9297 raft_consensus.cc:2229] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:22:03.404744 9297 raft_consensus.cc:2258] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:03.406168 9297 ts_tablet_manager.cc:1905] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:03.407917 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 8641
I20260807 18:22:03.408955 9296 ts_tablet_manager.cc:1918] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:03.409101 9296 log.cc:1196] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/6efd22bb60fb4e1a889a4ee85e0d92b1
I20260807 18:22:03.409525 9296 ts_tablet_manager.cc:1939] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P c0a59f504de14d1699431f305215cada: Deleting consensus metadata
I20260807 18:22:03.410454 9299 ts_tablet_manager.cc:1918] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:03.410606 9299 log.cc:1196] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/6efd22bb60fb4e1a889a4ee85e0d92b1
I20260807 18:22:03.410706 8521 catalog_manager.cc:4841] TS c0a59f504de14d1699431f305215cada (127.29.156.193:45739): tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (table test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]) successfully deleted
I20260807 18:22:03.411051 9299 ts_tablet_manager.cc:1939] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 312b1c66ca27488db65b9f5a775ed051: Deleting consensus metadata
I20260807 18:22:03.412642 8520 catalog_manager.cc:4841] TS 312b1c66ca27488db65b9f5a775ed051 (127.29.156.195:39023): tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (table test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]) successfully deleted
I20260807 18:22:03.417227 9297 ts_tablet_manager.cc:1918] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:22:03.417398 9297 log.cc:1196] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/6efd22bb60fb4e1a889a4ee85e0d92b1
I20260807 18:22:03.417860 9297 ts_tablet_manager.cc:1939] T 6efd22bb60fb4e1a889a4ee85e0d92b1 P 2cd4c058ecc9478c928c4edb6733ca93: Deleting consensus metadata
I20260807 18:22:03.419517 8520 catalog_manager.cc:4841] TS 2cd4c058ecc9478c928c4edb6733ca93 (127.29.156.194:41485): tablet 6efd22bb60fb4e1a889a4ee85e0d92b1 (table test-table [id=3c5a3e6998164df29e1c5785d65d3bbb]) successfully deleted
Aug 07 18:22:03 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.193@KRBTEST.COM
I20260807 18:22:03.435926 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.409737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46778 (local address 127.29.156.193:45739)
0807 18:22:03.420109 (+ 10372us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.420115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.420395 (+ 280us) server_negotiation.cc:407] Connection header received
0807 18:22:03.420668 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.420672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.420740 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.420861 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:03.422263 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.422993 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.424374 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.424811 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.424949 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.429377 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:03.429392 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:03.429408 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:03.429447 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:03.432701 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.433526 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.433534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.433536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.433679 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.434072 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.434116 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.434119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.434598 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:03.434913 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.435214 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.435451 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10239,"thread_start_us":10189,"threads_started":1}
I20260807 18:22:03.436703 8824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:03.584302 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 8641
I20260807 18:22:03.620903 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 8833
Aug 07 18:22:03 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.194@KRBTEST.COM
I20260807 18:22:03.641300 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.622009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37424 (local address 127.29.156.194:41485)
0807 18:22:03.622272 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.622279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.622301 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:22:03.622402 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.622408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.622521 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.622662 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:03.624754 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.626098 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.627588 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.628265 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.628429 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.633328 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:03.633350 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:03.633370 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:03.633415 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:03.637885 (+ 4470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.638699 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.638709 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.638712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.638871 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.639319 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.639326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.639329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.639977 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:03.640313 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.640639 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.640894 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:22:03.642198 9020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:03.811064 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 8833
I20260807 18:22:03.846011 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 9034
Aug 07 18:22:03 dist-test-slave-sb2z krb5kdc[8483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.156.195@KRBTEST.COM
I20260807 18:22:03.862498 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:03.846994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.29.156.195:39023)
0807 18:22:03.847654 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:03.847658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:03.847674 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:22:03.847731 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:03.847734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:03.847807 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:03.847896 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:03.849321 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.850044 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.851281 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:03.851789 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:03.851889 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:03.856439 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:03.856455 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:03.856471 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:03.856513 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:03.859909 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.860583 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.860590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.860592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.860719 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:03.861122 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:03.861126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:03.861128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:03.861487 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:03.861725 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:03.862021 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:22:03.862235 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":375,"threads_started":1}
I20260807 18:22:03.863240 9225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:03.993403 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 9034
I20260807 18:22:04.025796 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 8502
I20260807 18:22:04.027443 8603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:04.152046 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 8502
2026-08-07T18:22:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6170 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:22:04 dist-test-slave-sb2z krb5kdc[9399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:22:04 dist-test-slave-sb2z krb5kdc[9399](info): set up 2 sockets
Aug 07 18:22:04 dist-test-slave-sb2z krb5kdc[9399](info): commencing operation
krb5kdc: starting...
W20260807 18:22:05.490842 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:22:05 dist-test-slave-sb2z krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:22:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:22:05.800624 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41841
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:39329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:06.155170 9415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:06.155484 9415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:06.155679 9415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:06.163100 9415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:06.163213 9415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:06.163303 9415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:06.163424 9415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:06.163507 9415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:06.173712 9415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:39329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:06.175122 9415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:06.176489 9415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:06.189832 9421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:06.189774 9423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:06.189909 9425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:06.190603 9415 server_base.cc:1034] running on GCE node
I20260807 18:22:06.191601 9415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:06.193153 9415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:06.194442 9415 hybrid_clock.cc:648] HybridClock initialized: now 1786126926194386 us; error 40 us; skew 500 ppm
I20260807 18:22:06.194872 9415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:22:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2120 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:22:06 dist-test-slave-sb2z krb5kdc[9434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:22:06 dist-test-slave-sb2z krb5kdc[9434](info): set up 2 sockets
Aug 07 18:22:06 dist-test-slave-sb2z krb5kdc[9434](info): commencing operation
krb5kdc: starting...
W20260807 18:22:07.590049 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:22:07 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:22:07.907467 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:39059
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:39059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:08.264952 9451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:08.265328 9451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:08.265523 9451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:08.272907 9451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:08.273088 9451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:08.273212 9451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:08.273313 9451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:08.273399 9451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:08.283931 9451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:39059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:39059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:08.285377 9451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:08.286751 9451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:08.299155 9456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:08.299088 9459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:08.299127 9457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:08.300273 9451 server_base.cc:1034] running on GCE node
I20260807 18:22:08.301146 9451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:08.302629 9451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:08.303922 9451 hybrid_clock.cc:648] HybridClock initialized: now 1786126928303850 us; error 50 us; skew 500 ppm
I20260807 18:22:08.304317 9451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:08 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:08.311084 9451 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:22:08.312821 9451 webserver.cc:460] Webserver started at http://127.29.156.254:34359/ using document root <none> and password file <none>
I20260807 18:22:08.313551 9451 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:08.313658 9451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:08.313952 9451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:08.317205 9451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "8db3633ae9d946138a23258b40ca3d9c"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "4dfa9d6ed797c23037f8bba7cdbbc6d9"
server_key_iv: "61e9b6fb02205eec45a1fbb1ec6dda12"
server_key_version: "encryptionkey@0"
I20260807 18:22:08.317967 9451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "8db3633ae9d946138a23258b40ca3d9c"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "4dfa9d6ed797c23037f8bba7cdbbc6d9"
server_key_iv: "61e9b6fb02205eec45a1fbb1ec6dda12"
server_key_version: "encryptionkey@0"
I20260807 18:22:08.323721 9451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:22:08.327486 9470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:08.329125 9451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:08.329332 9451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "8db3633ae9d946138a23258b40ca3d9c"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "4dfa9d6ed797c23037f8bba7cdbbc6d9"
server_key_iv: "61e9b6fb02205eec45a1fbb1ec6dda12"
server_key_version: "encryptionkey@0"
I20260807 18:22:08.329576 9451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:08.350193 9451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:08.355072 9451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:08.355466 9451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:08.377189 9451 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:39059
I20260807 18:22:08.377228 9541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:39059 every 8 connection(s)
I20260807 18:22:08.378814 9451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:08.380324 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 9451
I20260807 18:22:08.380582 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:22:08.381323 30323 external_mini_cluster.cc:1428] Setting key 67d0b744fdbde81a1dd2918de791ecf3
I20260807 18:22:08.383600 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:22:08 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:08.397702 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Bootstrap starting.
I20260807 18:22:08.401998 9542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:08.403345 9542 log.cc:824] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:08.406940 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: No bootstrap required, opened a new log
I20260807 18:22:08.408017 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:08.383664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.29.156.254:39059)
0807 18:22:08.384561 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:08.384577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:08.384627 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:22:08.386876 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:08.386906 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:08.387444 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:08.387941 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:08.389554 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:08.391690 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:08.393145 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:08.394244 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:08.394443 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:08.398600 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:08.398610 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:08.398628 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:08.398674 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:08.402882 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:08.403664 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:08.403672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:08.403681 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:08.403909 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:08.404401 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:08.404404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:08.404406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:08.405123 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:08.405437 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:08.406069 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 18:22:08.406531 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":361,"threads_started":1}
I20260807 18:22:08.415454 9542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:08.415866 9542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:08.415943 9542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db3633ae9d946138a23258b40ca3d9c, State: Initialized, Role: FOLLOWER
I20260807 18:22:08.416520 9542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:08.416733 9542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:08.416887 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:08.417088 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:08.419406 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:08.419989 9542 leader_election.cc:302] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8db3633ae9d946138a23258b40ca3d9c; no voters:
I20260807 18:22:08.420606 9542 leader_election.cc:288] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:22:08.420783 9549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:08.421499 9549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 LEADER]: Becoming Leader. State: Replica: 8db3633ae9d946138a23258b40ca3d9c, State: Running, Role: LEADER
I20260807 18:22:08.422076 9549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:08.422870 9542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:08.425149 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8db3633ae9d946138a23258b40ca3d9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } } }
I20260807 18:22:08.425495 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:08.425956 9551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db3633ae9d946138a23258b40ca3d9c. Latest consensus state: current_term: 1 leader_uuid: "8db3633ae9d946138a23258b40ca3d9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } } }
I20260807 18:22:08.426281 9551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:08.427086 9555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:08.432005 9555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:08.441787 9555 catalog_manager.cc:1347] Generated new cluster ID: 3983c93bacfa4b569df930f7cf57d0d2
I20260807 18:22:08.441906 9555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:08.470706 9555 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:22:08.472280 9555 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:08.485677 9555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Generated new TSK 0
I20260807 18:22:08.486649 9555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:22:08.563452 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:08.935963 9582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:08.936316 9582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:08.936430 9582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:08.944981 9582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:08.945130 9582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:08.945263 9582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:08.956461 9582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:08.957793 9582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:08.959146 9582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:08.973011 9589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:08.973263 9591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:08.972994 9593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:08.974617 9582 server_base.cc:1034] running on GCE node
I20260807 18:22:08.975385 9582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:08.976567 9582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:08.977859 9582 hybrid_clock.cc:648] HybridClock initialized: now 1786126928977817 us; error 57 us; skew 500 ppm
I20260807 18:22:08.978295 9582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:08 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:08.982054 9582 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:22:08.983996 9582 webserver.cc:460] Webserver started at http://127.29.156.193:42405/ using document root <none> and password file <none>
I20260807 18:22:08.984843 9582 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:08.985028 9582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:08.985517 9582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:08.989158 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "263bb2402e334cd39df6225304bd9f18"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "927af989cd40d4015f4091f6f99929c3"
server_key_iv: "9ce83ec9021113d568b380798782ec1a"
server_key_version: "encryptionkey@0"
I20260807 18:22:08.990119 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "263bb2402e334cd39df6225304bd9f18"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "927af989cd40d4015f4091f6f99929c3"
server_key_iv: "9ce83ec9021113d568b380798782ec1a"
server_key_version: "encryptionkey@0"
I20260807 18:22:08.996119 9582 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:22:09.000120 9607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.001906 9582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:22:09.002178 9582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "263bb2402e334cd39df6225304bd9f18"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "927af989cd40d4015f4091f6f99929c3"
server_key_iv: "9ce83ec9021113d568b380798782ec1a"
server_key_version: "encryptionkey@0"
I20260807 18:22:09.002429 9582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:09.018494 9582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:09.023794 9582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:09.024179 9582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:09.025426 9582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:09.027010 9582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:09.027153 9582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.027341 9582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:09.027436 9582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.068768 9582 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:39035
I20260807 18:22:09.068813 9769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:39035 every 8 connection(s)
I20260807 18:22:09.070233 9582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:09.076032 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 9582
I20260807 18:22:09.076246 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:22:09.076920 30323 external_mini_cluster.cc:1428] Setting key b850d3a3e76afe2b756abbdcd3b303e9
Aug 07 18:22:09 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:09.092890 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:09.073106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:57177 (local address 127.29.156.254:39059)
0807 18:22:09.073617 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:09.073622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:09.074740 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 18:22:09.077067 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:09.077073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:09.077181 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:09.077316 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:09.078995 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.079853 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.081619 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.082066 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.082214 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:09.086207 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:09.086212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:09.086215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:09.086250 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:09.089211 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:09.090042 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:09.090049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:09.090053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:09.090197 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:09.090857 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:09.090862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:09.090864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:09.091165 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:09.091310 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:09.092410 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0807 18:22:09.092603 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":261,"threads_started":1}
I20260807 18:22:09.094638 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:09.073576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:57177)
0807 18:22:09.074498 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:09.074557 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:09.076748 (+ 2191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:09.077584 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:09.077593 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:09.078193 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:09.078806 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:09.078824 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.080043 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.080054 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:09.081451 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:09.081465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.082331 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.082338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:09.082594 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:09.083416 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:09.083441 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:09.085940 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:09.089390 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:09.089400 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:09.089427 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:09.089895 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:09.090326 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:09.090331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:09.090335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:09.090659 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:09.091320 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:09.091331 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:09.091907 (+ 576us) client_negotiation.cc:769] Sending connection context
0807 18:22:09.092292 (+ 385us) client_negotiation.cc:240] Negotiation successful
0807 18:22:09.092957 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":430,"threads_started":1}
I20260807 18:22:09.096508 9770 heartbeater.cc:347] Connected to a master server at 127.29.156.254:39059
I20260807 18:22:09.096885 9770 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:09.097955 9770 heartbeater.cc:511] Master 127.29.156.254:39059 requested a full tablet report, sending...
I20260807 18:22:09.100857 9495 ts_manager.cc:194] Registered new tserver with Master: 263bb2402e334cd39df6225304bd9f18 (127.29.156.193:39035)
I20260807 18:22:09.103452 9495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:57177
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:22:09.151548 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:09.525074 9778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:09.525357 9778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:09.525532 9778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:09.533191 9778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:09.533308 9778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:09.533515 9778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:09.543915 9778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:09.545284 9778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:09.546725 9778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:09.561228 9793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:09.561352 9791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:09.561921 9790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:09.561993 9778 server_base.cc:1034] running on GCE node
I20260807 18:22:09.562760 9778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:09.563951 9778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:09.565218 9778 hybrid_clock.cc:648] HybridClock initialized: now 1786126929565148 us; error 63 us; skew 500 ppm
I20260807 18:22:09.565604 9778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:09 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:09.569242 9778 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:22:09.570842 9778 webserver.cc:460] Webserver started at http://127.29.156.194:43375/ using document root <none> and password file <none>
I20260807 18:22:09.571626 9778 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:09.571738 9778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:09.572033 9778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:09.575040 9778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "174c7c4b1e5145f4a25ff066ef93820a"
format_stamp: "Formatted at 2026-08-07 18:22:09 on dist-test-slave-sb2z"
server_key: "c8b1cca51a6842b11221abc33ad3ab14"
server_key_iv: "a1261b016a57d817d44034183bbc5d59"
server_key_version: "encryptionkey@0"
I20260807 18:22:09.575893 9778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "174c7c4b1e5145f4a25ff066ef93820a"
format_stamp: "Formatted at 2026-08-07 18:22:09 on dist-test-slave-sb2z"
server_key: "c8b1cca51a6842b11221abc33ad3ab14"
server_key_iv: "a1261b016a57d817d44034183bbc5d59"
server_key_version: "encryptionkey@0"
I20260807 18:22:09.581723 9778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:22:09.585726 9804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.587461 9778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:22:09.587697 9778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "174c7c4b1e5145f4a25ff066ef93820a"
format_stamp: "Formatted at 2026-08-07 18:22:09 on dist-test-slave-sb2z"
server_key: "c8b1cca51a6842b11221abc33ad3ab14"
server_key_iv: "a1261b016a57d817d44034183bbc5d59"
server_key_version: "encryptionkey@0"
I20260807 18:22:09.588023 9778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:09.606812 9778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:09.611938 9778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:09.612327 9778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:09.613373 9778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:09.614822 9778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:09.614938 9778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.615114 9778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:09.615192 9778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:09.653455 9778 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:45261
I20260807 18:22:09.653529 9959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:45261 every 8 connection(s)
I20260807 18:22:09.654858 9778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:09.658913 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 9778
I20260807 18:22:09.659169 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:22:09.659745 30323 external_mini_cluster.cc:1428] Setting key e29be68f3042689b380b81e910f9813e
Aug 07 18:22:09 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:09.677035 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:09.657731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:55591 (local address 127.29.156.254:39059)
0807 18:22:09.658202 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:09.658207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:09.659349 (+ 1142us) server_negotiation.cc:407] Connection header received
0807 18:22:09.661487 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:09.661492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:09.661593 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:09.661731 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:09.663402 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.664224 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.666052 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.666503 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.666652 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:09.670371 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:09.670377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:09.670380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:09.670422 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:09.673295 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:09.674120 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:09.674128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:09.674133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:09.674275 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:09.674786 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:09.674792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:09.674796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:09.675222 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:09.675414 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:09.676241 (+ 827us) server_negotiation.cc:299] Negotiation successful
0807 18:22:09.676518 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260807 18:22:09.677693 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:09.658306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:55591)
0807 18:22:09.659170 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:09.659218 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:09.661212 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:09.662010 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:09.662019 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:09.662610 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:09.663209 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:09.663228 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.664504 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.664511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:09.665823 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:09.665837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:09.666756 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:09.666764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:09.667030 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:09.667801 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:09.667830 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:09.670130 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:09.673498 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:09.673508 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:09.673526 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:09.673986 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:09.674454 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:09.674458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:09.674460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:09.674663 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:09.675406 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:09.675412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:09.675831 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 18:22:09.676110 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 18:22:09.676526 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":375,"threads_started":1}
I20260807 18:22:09.679566 9961 heartbeater.cc:347] Connected to a master server at 127.29.156.254:39059
I20260807 18:22:09.679941 9961 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:09.680806 9961 heartbeater.cc:511] Master 127.29.156.254:39059 requested a full tablet report, sending...
I20260807 18:22:09.682493 9495 ts_manager.cc:194] Registered new tserver with Master: 174c7c4b1e5145f4a25ff066ef93820a (127.29.156.194:45261)
I20260807 18:22:09.683383 9495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:55591
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:22:09.732721 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:10.087231 9968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:10.087679 9968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:10.087805 9968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:10.095672 9968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:10.095804 9968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:10.095927 9968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:10.106585 9968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:10.108137 9968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:10.108141 9770 heartbeater.cc:503] Master 127.29.156.254:39059 was elected leader, sending a full tablet report...
I20260807 18:22:10.110003 9968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:10.125566 9977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:10.125641 9979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:10.125535 9981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:10.132093 9968 server_base.cc:1034] running on GCE node
I20260807 18:22:10.133019 9968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:10.134418 9968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:10.135672 9968 hybrid_clock.cc:648] HybridClock initialized: now 1786126930135613 us; error 41 us; skew 500 ppm
I20260807 18:22:10.136089 9968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:10 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126930, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:10.139878 9968 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:22:10.141686 9968 webserver.cc:460] Webserver started at http://127.29.156.195:37363/ using document root <none> and password file <none>
I20260807 18:22:10.142613 9968 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:10.142794 9968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:10.143152 9968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:10.146661 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "bfe5be70053543e18ab727c86549c2b4"
format_stamp: "Formatted at 2026-08-07 18:22:10 on dist-test-slave-sb2z"
server_key: "ecf391bd93cb2e4dc16df05c00599044"
server_key_iv: "a18dbfa1adb6f4ba8d8d7a9cf033034c"
server_key_version: "encryptionkey@0"
I20260807 18:22:10.147629 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "bfe5be70053543e18ab727c86549c2b4"
format_stamp: "Formatted at 2026-08-07 18:22:10 on dist-test-slave-sb2z"
server_key: "ecf391bd93cb2e4dc16df05c00599044"
server_key_iv: "a18dbfa1adb6f4ba8d8d7a9cf033034c"
server_key_version: "encryptionkey@0"
I20260807 18:22:10.153630 9968 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:22:10.158044 9992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:10.159687 9968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:10.159890 9968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "bfe5be70053543e18ab727c86549c2b4"
format_stamp: "Formatted at 2026-08-07 18:22:10 on dist-test-slave-sb2z"
server_key: "ecf391bd93cb2e4dc16df05c00599044"
server_key_iv: "a18dbfa1adb6f4ba8d8d7a9cf033034c"
server_key_version: "encryptionkey@0"
I20260807 18:22:10.160122 9968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:10.180290 9968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:10.184875 9968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:10.185194 9968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:10.186261 9968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:10.187908 9968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:10.188021 9968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:10.188181 9968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:10.188256 9968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:10.225921 9968 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:42417
I20260807 18:22:10.226016 10160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:42417 every 8 connection(s)
I20260807 18:22:10.227226 9968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:10.235682 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 9968
I20260807 18:22:10.235867 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:22:10.236364 30323 external_mini_cluster.cc:1428] Setting key c6d9bb97b9e10467eb47da762a73ba6e
Aug 07 18:22:10 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126930, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:10.249267 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:10.230055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:58377 (local address 127.29.156.254:39059)
0807 18:22:10.230565 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:10.230570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:10.231663 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 18:22:10.233537 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:10.233542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:10.233633 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:10.233784 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:10.235392 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.236440 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.238358 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.238891 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.239026 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:10.243078 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:10.243084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:10.243087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:10.243127 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:10.246153 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.246929 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.246934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.246936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.247134 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.247629 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.247633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.247635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.247907 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:10.248093 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:10.248774 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 18:22:10.248958 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":253,"threads_started":1}
I20260807 18:22:10.250283 10164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:10.230700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:58377)
0807 18:22:10.231431 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:10.231469 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:10.233254 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:10.234036 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:10.234045 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:10.234588 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:10.235147 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:10.235215 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.236640 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.236644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:10.238196 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:10.238209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.239166 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.239171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:10.239504 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:10.240286 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:10.240315 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:10.242765 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:10.246314 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:10.246322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:10.246335 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:10.246788 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:10.247252 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:10.247256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:10.247258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:10.247448 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:10.248063 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:10.248069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:10.248400 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 18:22:10.248644 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:22:10.249055 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":295,"threads_started":1}
I20260807 18:22:10.252031 10161 heartbeater.cc:347] Connected to a master server at 127.29.156.254:39059
I20260807 18:22:10.252388 10161 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:10.253170 10161 heartbeater.cc:511] Master 127.29.156.254:39059 requested a full tablet report, sending...
I20260807 18:22:10.254745 9494 ts_manager.cc:194] Registered new tserver with Master: bfe5be70053543e18ab727c86549c2b4 (127.29.156.195:42417)
I20260807 18:22:10.255831 9494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:58377
I20260807 18:22:10.265753 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
Entry for principal oryx/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab.
WARNING: no policy specified for oryx/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
Entry for principal oryx/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab.
WARNING: no policy specified for oryx/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
Entry for principal oryx/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab.
I20260807 18:22:10.468685 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 9582
Aug 07 18:22:10 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:22:10.491233 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:10.469927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.29.156.193:39035)
0807 18:22:10.470995 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:10.471009 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:10.471053 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:22:10.471148 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:10.471161 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:10.471555 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:10.471758 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:10.473210 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.475087 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.476503 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.477693 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.477863 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:10.482525 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:10.482547 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:10.482572 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:10.482629 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:10.487447 (+ 4818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.488356 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.488369 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.488375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.488577 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.489066 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.489074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.489079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.489648 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:10.490033 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:10.490299 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:22:10.490659 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":645,"threads_started":1}
I20260807 18:22:10.493067 9762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:10.579670 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 9582
I20260807 18:22:10.606029 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 9778
Aug 07 18:22:10 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:22:10.623920 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:10.607028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46946 (local address 127.29.156.194:45261)
0807 18:22:10.607631 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:10.607640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:10.607669 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:22:10.607733 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:10.607739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:10.607971 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:10.608112 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:10.609370 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.610552 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.611934 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.612664 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.612817 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:10.617415 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:10.617429 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:10.617445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:10.617484 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:10.620723 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.621573 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.621581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.621583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.621708 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.622175 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.622179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.622181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.622548 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:10.622912 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:10.623201 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:22:10.623477 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":296,"threads_started":1}
I20260807 18:22:10.625329 9954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:10.710557 9961 heartbeater.cc:503] Master 127.29.156.254:39059 was elected leader, sending a full tablet report...
I20260807 18:22:10.712425 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 9778
W20260807 18:22:10.735705 9477 connection.cc:535] server connection from 127.29.156.194:55591 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:22:10.736063 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 9968
Aug 07 18:22:10 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:22:10.754571 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:10.737239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.29.156.195:42417)
0807 18:22:10.737804 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:10.737811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:10.737844 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:22:10.737914 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:10.737921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:10.738148 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:10.738342 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:10.739894 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.740904 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.742276 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:10.742866 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:10.743022 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:10.748001 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:10.748016 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:10.748031 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:10.748071 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:10.751144 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.752162 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.752170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.752173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.752314 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:10.752750 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:10.752754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:10.752756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:10.753153 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:10.753483 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:10.753910 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 18:22:10.754211 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":291,"threads_started":1}
I20260807 18:22:10.756134 10155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:10.842849 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 9968
I20260807 18:22:10.868736 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 9451
I20260807 18:22:10.870150 9536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:10.988183 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 9451
I20260807 18:22:11.017473 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:39059
--webserver_interface=127.29.156.254
--webserver_port=34359
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:39059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:11.365288 10183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:11.365597 10183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:11.365762 10183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:11.373250 10183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:11.373379 10183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:11.373427 10183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:11.373549 10183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:11.373610 10183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:11.383962 10183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:39059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:39059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=34359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:11.385416 10183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:11.386794 10183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:11.399091 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:11.399166 10195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:11.399163 10190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:11.400496 10183 server_base.cc:1034] running on GCE node
I20260807 18:22:11.401288 10183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:11.402729 10183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:11.403982 10183 hybrid_clock.cc:648] HybridClock initialized: now 1786126931403941 us; error 51 us; skew 500 ppm
I20260807 18:22:11.404362 10183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126931, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:11.407724 10183 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:22:11.409209 10183 webserver.cc:460] Webserver started at http://127.29.156.254:34359/ using document root <none> and password file <none>
I20260807 18:22:11.409929 10183 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:11.410095 10183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:11.416270 10183 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:22:11.419394 10205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:11.421013 10183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:22:11.421240 10183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "8db3633ae9d946138a23258b40ca3d9c"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "4dfa9d6ed797c23037f8bba7cdbbc6d9"
server_key_iv: "61e9b6fb02205eec45a1fbb1ec6dda12"
server_key_version: "encryptionkey@0"
I20260807 18:22:11.422333 10183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:11.436169 10183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:11.440733 10183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:11.441051 10183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:11.461097 10183 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:39059
I20260807 18:22:11.461129 10270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:39059 every 8 connection(s)
I20260807 18:22:11.462674 10183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:11.467819 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 10183
I20260807 18:22:11.468875 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:39035
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:39059
--webserver_port=42405
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:22:11.470834 10271 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:22:11.476073 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Bootstrap starting.
I20260807 18:22:11.497097 10271 log.cc:824] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:11.506300 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:22:11.506886 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Bootstrap complete.
I20260807 18:22:11.516515 10271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:11.517349 10271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db3633ae9d946138a23258b40ca3d9c, State: Initialized, Role: FOLLOWER
I20260807 18:22:11.518179 10271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:11.518428 10271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:11.518544 10271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:11.518746 10271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:22:11.522708 10271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:11.523226 10271 leader_election.cc:302] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8db3633ae9d946138a23258b40ca3d9c; no voters:
I20260807 18:22:11.523968 10271 leader_election.cc:288] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:22:11.524173 10277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:22:11.525465 10277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [term 2 LEADER]: Becoming Leader. State: Replica: 8db3633ae9d946138a23258b40ca3d9c, State: Running, Role: LEADER
I20260807 18:22:11.526153 10277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } }
I20260807 18:22:11.526432 10271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:11.529091 10279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db3633ae9d946138a23258b40ca3d9c. Latest consensus state: current_term: 2 leader_uuid: "8db3633ae9d946138a23258b40ca3d9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } } }
I20260807 18:22:11.529469 10279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:11.529999 10278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8db3633ae9d946138a23258b40ca3d9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db3633ae9d946138a23258b40ca3d9c" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 39059 } } }
I20260807 18:22:11.530360 10278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:11.532025 10282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:11.537778 10282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:11.539232 10282 catalog_manager.cc:1259] Loaded cluster ID: 3983c93bacfa4b569df930f7cf57d0d2
I20260807 18:22:11.539337 10282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:11.543438 10282 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:11.545315 10282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8db3633ae9d946138a23258b40ca3d9c: Loaded TSK: 0
I20260807 18:22:11.546846 10282 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:22:11.842877 10275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:11.843264 10275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:11.843449 10275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:11.851266 10275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:11.851455 10275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:11.851679 10275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:11.861987 10275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.193.keytab
--principal=oryx/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:39035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=42405
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:11.863396 10275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:11.864776 10275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:11.880070 10308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:11.879900 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:11.880815 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:11.880963 10275 server_base.cc:1034] running on GCE node
I20260807 18:22:11.881699 10275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:11.882937 10275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:11.884168 10275 hybrid_clock.cc:648] HybridClock initialized: now 1786126931884121 us; error 51 us; skew 500 ppm
I20260807 18:22:11.884550 10275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126931, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:11.888908 10275 init.cc:377] Logged in from keytab as oryx/127.29.156.193@KRBTEST.COM (short username oryx)
I20260807 18:22:11.890820 10275 webserver.cc:460] Webserver started at http://127.29.156.193:42405/ using document root <none> and password file <none>
I20260807 18:22:11.891757 10275 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:11.891928 10275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:11.899245 10275 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:22:11.902428 10319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:11.904145 10275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:11.904312 10275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "263bb2402e334cd39df6225304bd9f18"
format_stamp: "Formatted at 2026-08-07 18:22:08 on dist-test-slave-sb2z"
server_key: "927af989cd40d4015f4091f6f99929c3"
server_key_iv: "9ce83ec9021113d568b380798782ec1a"
server_key_version: "encryptionkey@0"
I20260807 18:22:11.905323 10275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:11.924686 10275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:11.929629 10275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:11.929872 10275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:11.930850 10275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:11.932282 10275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:11.932382 10275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:11.932471 10275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:11.932540 10275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:11.969163 10275 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:39035
I20260807 18:22:11.969223 10467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:39035 every 8 connection(s)
I20260807 18:22:11.970638 10275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:11.971817 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 10275
I20260807 18:22:11.972622 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:45261
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:39059
--webserver_port=43375
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:11.993029 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:11.974246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:40395)
0807 18:22:11.975164 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:11.975211 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:11.977025 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:11.981402 (+ 4377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:11.981415 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:11.982098 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:11.982646 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:11.982665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.984409 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.984413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:11.985702 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:11.985715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.986780 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.986788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:11.987081 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:11.987818 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:11.987843 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:11.991636 (+ 3793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":398,"threads_started":1}
W20260807 18:22:11.993072 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:11.974302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40395 (local address 127.29.156.254:39059)
0807 18:22:11.980025 (+ 5723us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:11.980038 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:11.980080 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:22:11.980396 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:11.980409 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:11.980758 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:11.981256 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:11.982860 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.984224 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.985867 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.986531 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.986741 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:11.992003 (+ 5262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:40395: BlockingRecv error: recv got EOF from 127.29.156.193:40395 (error 108)
Metrics: {"server-negotiator.queue_time_us":5477,"thread_start_us":5293,"threads_started":1}
W20260807 18:22:11.993921 10469 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:39059 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.156.254:39059: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:11 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:12.002912 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:11.994668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:36585)
0807 18:22:11.994893 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:11.994916 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:11.995035 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:11.995346 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:11.995353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:11.995891 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:11.996118 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:11.996133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.997236 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.997241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:11.998155 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:11.998167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.998856 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.998862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:11.999045 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:11.999645 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:11.999663 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.002617 (+ 2954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:22:12.003364 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:11.994871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:36585 (local address 127.29.156.254:39059)
0807 18:22:11.995040 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:11.995044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:11.995061 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:22:11.995146 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:11.995149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:11.995207 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:11.995283 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:11.996286 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.997082 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.998305 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:11.998701 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:11.998830 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.002810 (+ 3980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:36585: BlockingRecv error: recv got EOF from 127.29.156.193:36585 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:12.012579 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.003728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:59819)
0807 18:22:12.003916 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.003938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.004060 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.004392 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.004397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.004771 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:12.004916 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.004923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.005901 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.005906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.006815 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.006826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.007896 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.007903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.008273 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.009057 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:12.009077 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.012227 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:22:12.012733 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.003798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:59819 (local address 127.29.156.254:39059)
0807 18:22:12.003941 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.003947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.003989 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:22:12.004180 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.004186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.004265 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.004367 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.005057 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.005749 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.006957 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.007719 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.007897 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.012461 (+ 4564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:59819: BlockingRecv error: recv got EOF from 127.29.156.193:59819 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:22:12.012931 10469 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:22:12.330452 10473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:12.330812 10473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:12.330999 10473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:12.338634 10473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:12.338805 10473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:12.339006 10473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:12.349442 10473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.194.keytab
--principal=oryx/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:45261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=43375
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:12.350906 10473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:12.352409 10473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:12.366986 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:12.366950 10485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:12.367147 10482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:12.367808 10473 server_base.cc:1034] running on GCE node
I20260807 18:22:12.368467 10473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:12.369513 10473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:12.370728 10473 hybrid_clock.cc:648] HybridClock initialized: now 1786126932370686 us; error 33 us; skew 500 ppm
I20260807 18:22:12.371019 10473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126932, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:12.374455 10473 init.cc:377] Logged in from keytab as oryx/127.29.156.194@KRBTEST.COM (short username oryx)
I20260807 18:22:12.376173 10473 webserver.cc:460] Webserver started at http://127.29.156.194:43375/ using document root <none> and password file <none>
I20260807 18:22:12.376933 10473 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:12.377108 10473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:12.383144 10473 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:22:12.386360 10493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.388231 10473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:22:12.388443 10473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "174c7c4b1e5145f4a25ff066ef93820a"
format_stamp: "Formatted at 2026-08-07 18:22:09 on dist-test-slave-sb2z"
server_key: "c8b1cca51a6842b11221abc33ad3ab14"
server_key_iv: "a1261b016a57d817d44034183bbc5d59"
server_key_version: "encryptionkey@0"
I20260807 18:22:12.389519 10473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:12.419090 10473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:12.423913 10473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:12.424235 10473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:12.425350 10473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:12.427135 10473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:12.427269 10473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.427392 10473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:12.427498 10473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.466058 10473 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:45261
I20260807 18:22:12.466162 10643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:45261 every 8 connection(s)
I20260807 18:22:12.467621 10473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:12.477555 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 10473
I20260807 18:22:12.478376 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:42417
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:39059
--webserver_port=37363
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:22:12.488230 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.470568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:44725 (local address 127.29.156.254:39059)
0807 18:22:12.470793 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.470800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.472126 (+ 1326us) server_negotiation.cc:407] Connection header received
0807 18:22:12.474388 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.474392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.474463 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.474557 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.476394 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.477198 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.479831 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.480265 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.480374 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.487987 (+ 7613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:44725: BlockingRecv error: recv got EOF from 127.29.156.194:44725 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:22:12.489372 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.471076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:44725)
0807 18:22:12.471928 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.471982 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.474127 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.474824 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.474833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.475456 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:12.476178 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.476198 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.477427 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.477433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.479650 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.479673 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.480631 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.480644 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.481163 (+ 519us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.482064 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:12.482105 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.487796 (+ 5691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":310,"threads_started":1}
W20260807 18:22:12.490136 10644 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:39059 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.156.254:39059: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:12.499296 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.490814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:48471)
0807 18:22:12.491015 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.491038 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.491204 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.491619 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.491624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.491988 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:12.492145 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.492157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.493323 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.493327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.494459 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.494477 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.495212 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.495219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.495400 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.496097 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:12.496115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.498979 (+ 2864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:22:12.499445 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.490957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:48471 (local address 127.29.156.254:39059)
0807 18:22:12.491136 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.491142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.491161 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:22:12.491355 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.491361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.491436 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.491598 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.492296 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.493161 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.494588 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.495039 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.495190 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.499205 (+ 4015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:48471: BlockingRecv error: recv got EOF from 127.29.156.194:48471 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:12.512410 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.500075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:35495)
0807 18:22:12.500899 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.500924 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.501083 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.501448 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.501456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.502134 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:12.502520 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.502551 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.503813 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.503823 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.505857 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.505880 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.506695 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.506706 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.506953 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.507808 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:12.507841 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.512038 (+ 4197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":355,"threads_started":1}
W20260807 18:22:12.512491 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.500197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35495 (local address 127.29.156.254:39059)
0807 18:22:12.500440 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.500449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.501022 (+ 573us) server_negotiation.cc:407] Connection header received
0807 18:22:12.501213 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.501217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.501280 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.501365 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.502748 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.503661 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.506045 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.506480 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.506584 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.512267 (+ 5683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:35495: BlockingRecv error: recv got EOF from 127.29.156.194:35495 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:22:12.512703 10644 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:22:12.844674 10647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:12.845086 10647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:12.845263 10647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:12.852715 10647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:12.852882 10647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:12.853082 10647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:12.863183 10647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/krb5kdc/oryx_127.29.156.195.keytab
--principal=oryx/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:42417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=37363
--tserver_master_addrs=127.29.156.254:39059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:12.864643 10647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:12.866051 10647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:12.879853 10659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:12.879930 10661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:12.879983 10657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:12.881283 10647 server_base.cc:1034] running on GCE node
I20260807 18:22:12.881989 10647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:12.883159 10647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:12.884414 10647 hybrid_clock.cc:648] HybridClock initialized: now 1786126932884342 us; error 68 us; skew 500 ppm
I20260807 18:22:12.884765 10647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126932, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:12.888485 10647 init.cc:377] Logged in from keytab as oryx/127.29.156.195@KRBTEST.COM (short username oryx)
I20260807 18:22:12.890027 10647 webserver.cc:460] Webserver started at http://127.29.156.195:37363/ using document root <none> and password file <none>
I20260807 18:22:12.890800 10647 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:12.890959 10647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:12.897647 10647 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:22:12.900681 10672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.902338 10647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:12.902547 10647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "bfe5be70053543e18ab727c86549c2b4"
format_stamp: "Formatted at 2026-08-07 18:22:10 on dist-test-slave-sb2z"
server_key: "ecf391bd93cb2e4dc16df05c00599044"
server_key_iv: "a18dbfa1adb6f4ba8d8d7a9cf033034c"
server_key_version: "encryptionkey@0"
I20260807 18:22:12.903707 10647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:12.925666 10647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:12.930559 10647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:12.930871 10647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:12.932065 10647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:12.933579 10647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:12.933681 10647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.933835 10647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:12.933923 10647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:12.971639 10647 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:42417
I20260807 18:22:12.971702 10825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:42417 every 8 connection(s)
I20260807 18:22:12.973234 10647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:12.974985 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 10647
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:12 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:12.995862 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.976073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.29.156.254:39059)
0807 18:22:12.976278 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.976283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.977170 (+ 887us) server_negotiation.cc:407] Connection header received
0807 18:22:12.977442 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.977449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.977562 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.977694 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.980187 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.981525 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.982906 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.983663 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.983819 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.987129 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:12.987141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:12.987161 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:12.987209 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:12.991743 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:12.992820 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:12.992831 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:12.992841 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:12.993096 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:12.993566 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:12.993608 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:12.993611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:12.994221 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:12.994592 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:12.995055 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:22:12.995476 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:22:12.996198 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.977352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:39275 (local address 127.29.156.254:39059)
0807 18:22:12.979841 (+ 2489us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.979847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.979868 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:22:12.982136 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.982142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.982236 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.982393 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:12.984755 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.986063 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.988274 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.988968 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.989172 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.995876 (+ 6704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:39275: BlockingRecv error: recv got EOF from 127.29.156.195:39275 (error 108)
Metrics: {"server-negotiator.queue_time_us":2366,"thread_start_us":2240,"threads_started":1}
I20260807 18:22:12.996843 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.978023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:39275)
0807 18:22:12.978911 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.978964 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.981833 (+ 2869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.982704 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.982713 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.983630 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:12.984514 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.984547 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.986268 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.986272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.988077 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:12.988090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:12.989308 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:12.989314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:12.989691 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:12.990365 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:12.990410 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:12.995307 (+ 4897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":378,"threads_started":1}
W20260807 18:22:12.997573 10826 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:39059 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.156.254:39059: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:13.007202 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:12.998269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:48745)
0807 18:22:12.998508 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:12.998530 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:12.998641 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:12.999327 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:12.999331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:12.999907 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:13.000099 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.000109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.001218 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.001225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.002133 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.002145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.002888 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.002894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.003102 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.003722 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:13.003741 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:13.006859 (+ 3118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:22:13.007387 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:12.998803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:48745 (local address 127.29.156.254:39059)
0807 18:22:12.998979 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:12.998983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:12.998999 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:22:12.999039 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:12.999042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:12.999151 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:12.999268 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:13.000242 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.001068 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.002268 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.002719 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.002871 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.007117 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:48745: BlockingRecv error: recv got EOF from 127.29.156.195:48745 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:13.017249 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:13.008084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:47641)
0807 18:22:13.008320 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:13.008343 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:13.008491 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:13.008799 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:13.008803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:13.009185 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:13.009369 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.009378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.010532 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.010537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.011897 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.011911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.012874 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.012931 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.013086 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.013815 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:13.013840 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:13.017019 (+ 3179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260807 18:22:13.017525 10472 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:13.008274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:47641 (local address 127.29.156.254:39059)
0807 18:22:13.008532 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:13.008537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:13.008554 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:22:13.008598 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:13.008601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:13.008698 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:13.008773 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:13.009530 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.010332 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.012067 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.012721 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.012822 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.017220 (+ 4398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:47641: BlockingRecv error: recv got EOF from 127.29.156.195:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
W20260807 18:22:13.017623 10826 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:13.024127 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:13.013622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:41879)
0807 18:22:13.014193 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:13.014210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:13.014324 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:13.014666 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:13.014669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:13.015171 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:13.015406 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.015421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.017503 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.017509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.018671 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.018686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.019493 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.019500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.019714 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.020321 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:13.020343 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:13.023780 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":310,"threads_started":1}
W20260807 18:22:13.024466 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:13.013711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:41879 (local address 127.29.156.254:39059)
0807 18:22:13.013912 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:13.013921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:13.014300 (+ 379us) server_negotiation.cc:407] Connection header received
0807 18:22:13.014441 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:13.014447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:13.014526 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:13.014639 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:13.015610 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.016323 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.018815 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.019274 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.019388 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.024069 (+ 4681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:41879: BlockingRecv error: recv got EOF from 127.29.156.193:41879 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:13 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:13.528998 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:13.513547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:42195)
0807 18:22:13.514183 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:13.514206 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:13.514395 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:13.515005 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:13.515010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:13.515702 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:13.515946 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.515956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.516993 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.516997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.518204 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:13.518219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.518991 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.518997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:13.519152 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.524727 (+ 5575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:13.524753 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:13.528649 (+ 3896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":314,"threads_started":1}
W20260807 18:22:13.529110 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:13.513625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:42195 (local address 127.29.156.254:39059)
0807 18:22:13.513895 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:13.513903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:13.514319 (+ 416us) server_negotiation.cc:407] Connection header received
0807 18:22:13.514591 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:13.514603 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:13.514726 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:13.514855 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:13.516091 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.516837 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.518338 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:13.518783 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:13.518881 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:13.528865 (+ 9984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:42195: BlockingRecv error: recv got EOF from 127.29.156.194:42195 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:14.028395 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:14.018421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:43397)
0807 18:22:14.019045 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:14.019062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:14.019236 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:14.019695 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:14.019699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:14.020231 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:14.020432 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.020441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.021656 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.021662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.022789 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.022801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.023593 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.023600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.023769 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.024357 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:14.024374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:14.028128 (+ 3754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":301,"threads_started":1}
W20260807 18:22:14.028656 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:14.018564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43397 (local address 127.29.156.254:39059)
0807 18:22:14.018809 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:14.018816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:14.019166 (+ 350us) server_negotiation.cc:407] Connection header received
0807 18:22:14.019374 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:14.019380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:14.019540 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:14.019659 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:14.020597 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.021506 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.022940 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.023366 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.023509 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.028383 (+ 4874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:43397: BlockingRecv error: recv got EOF from 127.29.156.195:43397 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:14.035156 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:14.025103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:51259)
0807 18:22:14.026109 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:14.026133 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:14.026296 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:14.026790 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:14.026797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:14.027408 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:14.027698 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.027711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.029104 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.029110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.030040 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.030053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.030782 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.030788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.030978 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.031601 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:14.031618 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:14.034745 (+ 3127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":652,"threads_started":1}
W20260807 18:22:14.035430 10837 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:14.025183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:51259 (local address 127.29.156.254:39059)
0807 18:22:14.025733 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:14.025739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:14.026247 (+ 508us) server_negotiation.cc:407] Connection header received
0807 18:22:14.026530 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:14.026534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:14.026623 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:14.026749 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:14.027842 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.028960 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.030188 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.030616 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.030749 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.035083 (+ 4334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:51259: BlockingRecv error: recv got EOF from 127.29.156.193:51259 (error 108)
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:14 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:14.539315 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:14.530079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:59095)
0807 18:22:14.530586 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:14.530604 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:14.530730 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:14.531095 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:14.531100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:14.531711 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:14.531921 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.531940 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.533096 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.533104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.534391 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:14.534406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.535201 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.535207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:14.535357 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.536126 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:14.536146 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:14.539073 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
W20260807 18:22:14.539508 10837 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:14.530273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:59095 (local address 127.29.156.254:39059)
0807 18:22:14.530520 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:14.530527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:14.530689 (+ 162us) server_negotiation.cc:407] Connection header received
0807 18:22:14.530841 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:14.530845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:14.530910 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:14.530997 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:14.532078 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.532952 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.534579 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:14.535026 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:14.535132 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:14.539290 (+ 4158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:59095: BlockingRecv error: recv got EOF from 127.29.156.194:59095 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:15.039456 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:15.029464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:38655)
0807 18:22:15.030190 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:15.030208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:15.030438 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:15.030832 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:15.030838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:15.031358 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:15.031623 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.031637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.033069 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.033075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.034000 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.034012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.034769 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.034776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.034960 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.035701 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:15.035721 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:15.039077 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":385,"threads_started":1}
W20260807 18:22:15.039646 10837 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:15.029665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:38655 (local address 127.29.156.254:39059)
0807 18:22:15.029948 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:15.029957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:15.030344 (+ 387us) server_negotiation.cc:407] Connection header received
0807 18:22:15.030613 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:15.030617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:15.030683 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:15.030794 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:15.031800 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.032919 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.034162 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.034607 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.034747 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.039298 (+ 4551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:38655: BlockingRecv error: recv got EOF from 127.29.156.195:38655 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:15.046478 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:15.036116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:49407)
0807 18:22:15.037265 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:15.037294 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:15.037473 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:15.037882 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:15.037887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:15.038299 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:15.038549 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.038561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.039877 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.039883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.041032 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.041046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.041823 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.041830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.042066 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.042739 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:15.042757 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:15.046215 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1014,"thread_start_us":788,"threads_started":1}
W20260807 18:22:15.046715 10841 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:15.036373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:49407 (local address 127.29.156.254:39059)
0807 18:22:15.036989 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:15.036998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:15.037423 (+ 425us) server_negotiation.cc:407] Connection header received
0807 18:22:15.037649 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:15.037653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:15.037716 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:15.037838 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:15.038717 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.039666 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.041198 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.041662 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.041808 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.046444 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:49407: BlockingRecv error: recv got EOF from 127.29.156.193:49407 (error 108)
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":293,"threads_started":1}
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:15 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:15.549337 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:15.540556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:36823)
0807 18:22:15.541043 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:15.541060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:15.541185 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:15.541489 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:15.541492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:15.541863 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:15.542027 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.542036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.543089 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.543093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.544223 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:15.544237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.544962 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.544970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:15.545111 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.545805 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:15.545825 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:15.549012 (+ 3187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":265,"threads_started":1}
W20260807 18:22:15.549634 10841 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:15.540717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:36823 (local address 127.29.156.254:39059)
0807 18:22:15.540958 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:15.540965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:15.541139 (+ 174us) server_negotiation.cc:407] Connection header received
0807 18:22:15.541291 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:15.541295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:15.541362 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:15.541445 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:15.542170 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.542955 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.544375 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:15.544799 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:15.544896 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:15.549323 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:36823: BlockingRecv error: recv got EOF from 127.29.156.194:36823 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:16.050909 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:16.040800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:60605)
0807 18:22:16.041492 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:16.041536 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:16.041700 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:16.042156 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:16.042164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:16.042736 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:16.042925 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.042935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.044108 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.044113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.045330 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.045343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.046018 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.046022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.046208 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.046888 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:16.046969 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:16.050509 (+ 3540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":333,"threads_started":1}
W20260807 18:22:16.050992 10841 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:16.040982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:60605 (local address 127.29.156.254:39059)
0807 18:22:16.041168 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:16.041175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:16.041664 (+ 489us) server_negotiation.cc:407] Connection header received
0807 18:22:16.041832 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:16.041835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:16.041967 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:16.042076 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:16.043083 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.043960 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.045468 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.045879 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.045974 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.050732 (+ 4758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:60605: BlockingRecv error: recv got EOF from 127.29.156.195:60605 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:16.057363 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:16.047496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:39383)
0807 18:22:16.048553 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:16.048578 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:16.048756 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:16.049161 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:16.049165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:16.049619 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:16.049833 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.049850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.051000 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.051008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.052175 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.052190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.052919 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.052925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.053107 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.053792 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:16.053819 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:16.057013 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":787,"threads_started":1}
W20260807 18:22:16.057544 10847 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:16.047660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:39383 (local address 127.29.156.254:39059)
0807 18:22:16.048353 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:16.048360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:16.048698 (+ 338us) server_negotiation.cc:407] Connection header received
0807 18:22:16.048898 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:16.048904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:16.048984 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:16.049107 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:16.050026 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.050838 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.052335 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.052752 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.052883 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.057234 (+ 4351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:39383: BlockingRecv error: recv got EOF from 127.29.156.193:39383 (error 108)
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":388,"threads_started":1}
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:16 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:16.560465 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:16.550400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:44275)
0807 18:22:16.550922 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:16.550939 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:16.551075 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:16.551445 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:16.551450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:16.552129 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:16.552389 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.552409 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.553518 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.553521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.554525 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:16.554537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.555437 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.555443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:16.555672 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.556434 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:16.556461 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:16.560210 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":305,"threads_started":1}
W20260807 18:22:16.560674 10847 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:16.550521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:44275 (local address 127.29.156.254:39059)
0807 18:22:16.550809 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:16.550816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:16.551075 (+ 259us) server_negotiation.cc:407] Connection header received
0807 18:22:16.551189 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:16.551193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:16.551280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:16.551391 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:16.552560 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.553361 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.554697 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:16.555250 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:16.555365 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:16.560424 (+ 5059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:44275: BlockingRecv error: recv got EOF from 127.29.156.194:44275 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:17.061585 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:17.052094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:56623)
0807 18:22:17.052669 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:17.052686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:17.052803 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:17.053115 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:17.053118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:17.053499 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:17.053674 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.053683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.054698 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.054702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.055702 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.055715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.056397 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.056401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.056525 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.057157 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:17.057176 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:17.061243 (+ 4067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
W20260807 18:22:17.061766 10847 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:17.052139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:56623 (local address 127.29.156.254:39059)
0807 18:22:17.052325 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:17.052330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:17.052761 (+ 431us) server_negotiation.cc:407] Connection header received
0807 18:22:17.052899 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:17.052903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:17.052967 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:17.053048 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:17.053818 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.054565 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.055835 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.056253 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.056351 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.061458 (+ 5107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:56623: BlockingRecv error: recv got EOF from 127.29.156.195:56623 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:17.068401 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:17.058398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:49097)
0807 18:22:17.059011 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:17.059032 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:17.059200 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:17.059653 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:17.059659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:17.060417 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:17.060631 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.060643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.062110 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.062116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.063135 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.063148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.064285 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.064290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.064425 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.065057 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:17.065075 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:17.068108 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":279,"threads_started":1}
W20260807 18:22:17.068635 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:17.058553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:49097 (local address 127.29.156.254:39059)
0807 18:22:17.059326 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:17.059330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:17.059346 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:22:17.059384 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:17.059388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:17.059479 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:17.059614 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:17.060799 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.061946 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.063307 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.063894 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.064006 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.068338 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:49097: BlockingRecv error: recv got EOF from 127.29.156.193:49097 (error 108)
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":507,"threads_started":1}
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:17 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:17.571347 10852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:17.561557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:58743)
0807 18:22:17.562470 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:17.562487 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:17.562681 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:17.563051 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:17.563056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:17.563503 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:17.563746 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.563756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.564828 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.564834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.565989 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:17.566024 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.567025 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.567030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:17.567164 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.567859 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:17.567876 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:17.570975 (+ 3099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":344,"threads_started":1}
W20260807 18:22:17.571415 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:17.561784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:58743 (local address 127.29.156.254:39059)
0807 18:22:17.562047 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:17.562055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:17.562591 (+ 536us) server_negotiation.cc:407] Connection header received
0807 18:22:17.562835 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:17.562839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:17.562904 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:17.563010 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:17.563899 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.564672 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.566195 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:17.566871 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:17.566998 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:17.571180 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:58743: BlockingRecv error: recv got EOF from 127.29.156.194:58743 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:18.072683 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:18.062712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:51081)
0807 18:22:18.063351 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:18.063367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:18.063490 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:18.063836 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:18.063839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:18.064278 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:18.064443 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.064452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.065457 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.065461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.066380 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.066392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.067313 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.067320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.067622 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.068329 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:18.068349 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:18.072323 (+ 3974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":332,"threads_started":1}
W20260807 18:22:18.072911 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:18.062914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:51081 (local address 127.29.156.254:39059)
0807 18:22:18.063156 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:18.063164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:18.063484 (+ 320us) server_negotiation.cc:407] Connection header received
0807 18:22:18.063633 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:18.063637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:18.063703 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:18.063779 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:18.064595 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.065318 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.066554 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.067116 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.067251 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.072552 (+ 5301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:51081: BlockingRecv error: recv got EOF from 127.29.156.195:51081 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:18.078824 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:18.069428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:50963)
0807 18:22:18.070136 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:18.070158 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:18.070281 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:18.070767 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:18.070775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:18.071237 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:18.071400 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.071409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.073028 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.073035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.074050 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.074063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.074779 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.074784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.074917 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.075647 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:18.075667 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:18.078576 (+ 2909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":323,"threads_started":1}
W20260807 18:22:18.078996 10855 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:18.069495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:50963 (local address 127.29.156.254:39059)
0807 18:22:18.070395 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:18.070404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:18.070426 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:22:18.070480 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:18.070488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:18.070595 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:18.070729 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:18.071593 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.072796 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.074205 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.074628 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.074727 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.078787 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:50963: BlockingRecv error: recv got EOF from 127.29.156.193:50963 (error 108)
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":608,"threads_started":1}
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:18 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:18.585660 10856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:18.572412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:39607)
0807 18:22:18.572992 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:18.573011 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:18.573200 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:18.573681 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:18.573685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:18.574297 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:18.574461 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.574471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.575454 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.575458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.576543 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:18.576556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.577241 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.577246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:18.577379 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.581944 (+ 4565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:18.581965 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:18.585402 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":337,"threads_started":1}
W20260807 18:22:18.585978 10855 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:18.572599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:39607 (local address 127.29.156.254:39059)
0807 18:22:18.572859 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:18.572866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:18.573099 (+ 233us) server_negotiation.cc:407] Connection header received
0807 18:22:18.573382 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:18.573392 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:18.573492 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:18.573615 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:18.574603 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.575322 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.576704 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:18.577087 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:18.577189 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:18.585647 (+ 8458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:39607: BlockingRecv error: recv got EOF from 127.29.156.194:39607 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:19.083669 10857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:19.073995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:37963)
0807 18:22:19.074666 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:19.074684 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:19.074807 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:19.075143 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:19.075146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:19.075574 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:19.075756 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.075764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.076833 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.076837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.077788 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.077801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.078668 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.078673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.078808 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.079682 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:19.079707 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:19.083310 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":373,"threads_started":1}
W20260807 18:22:19.083880 10855 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:19.074121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:37963 (local address 127.29.156.254:39059)
0807 18:22:19.074333 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:19.074338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:19.074771 (+ 433us) server_negotiation.cc:407] Connection header received
0807 18:22:19.074923 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:19.074927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:19.075000 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:19.075092 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:19.075888 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.076629 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.077961 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.078468 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.078579 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.083586 (+ 5007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:37963: BlockingRecv error: recv got EOF from 127.29.156.195:37963 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:19.089679 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:19.079910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:52769)
0807 18:22:19.080535 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:19.080559 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:19.080710 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:19.081133 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:19.081138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:19.081626 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:19.081807 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.081817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.083241 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.083249 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.084649 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.084664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.085416 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.085422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.085604 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.086276 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:19.086296 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:19.089284 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":288,"threads_started":1}
W20260807 18:22:19.089823 10861 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:19.079973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:52769 (local address 127.29.156.254:39059)
0807 18:22:19.080830 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:19.080837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:19.080857 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:22:19.080917 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:19.080920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:19.080971 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:19.081088 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:19.081971 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.083086 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.084809 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.085253 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.085391 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.089526 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:52769: BlockingRecv error: recv got EOF from 127.29.156.193:52769 (error 108)
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":581,"threads_started":1}
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:19 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:19.596779 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:19.586813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:32869)
0807 18:22:19.587474 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:19.587490 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:19.587681 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:19.588126 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:19.588129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:19.588551 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:19.588739 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.588747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.589889 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.589893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.590986 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:19.591000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.591895 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.591900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:19.592071 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.592736 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:19.592790 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:19.596524 (+ 3734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":311,"threads_started":1}
W20260807 18:22:19.596879 10861 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:19.586898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:32869 (local address 127.29.156.254:39059)
0807 18:22:19.587100 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:19.587105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:19.587603 (+ 498us) server_negotiation.cc:407] Connection header received
0807 18:22:19.587852 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:19.587856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:19.587925 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:19.588068 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:19.588889 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.589741 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.591160 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:19.591712 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:19.591820 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:19.596702 (+ 4882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:32869: BlockingRecv error: recv got EOF from 127.29.156.194:32869 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:20.095382 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:20.084792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:43721)
0807 18:22:20.085438 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:20.085456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:20.085582 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:20.085962 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:20.085965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:20.086382 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:20.086549 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.086558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.087808 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.087812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.089062 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.089076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.090199 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.090208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.090380 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.091282 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:20.091316 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:20.094992 (+ 3676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":313,"threads_started":1}
W20260807 18:22:20.095516 10861 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:20.084966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43721 (local address 127.29.156.254:39059)
0807 18:22:20.085172 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:20.085176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:20.085556 (+ 380us) server_negotiation.cc:407] Connection header received
0807 18:22:20.085694 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:20.085703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:20.085798 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:20.085892 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:20.086731 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.087607 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.089318 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.090017 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.090166 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.095251 (+ 5085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:43721: BlockingRecv error: recv got EOF from 127.29.156.195:43721 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:20.100289 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:20.090690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:58559)
0807 18:22:20.091344 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:20.091366 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:20.091509 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:20.091905 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:20.091910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:20.092338 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:20.092497 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.092506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.093873 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.093878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.095334 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.095353 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.096296 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.096300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.096419 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.097037 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:20.097059 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:20.100047 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":378,"threads_started":1}
W20260807 18:22:20.100587 10867 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:20.090935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:58559 (local address 127.29.156.254:39059)
0807 18:22:20.091620 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:20.091625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:20.091643 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:22:20.091696 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:20.091699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:20.091753 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:20.091867 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:20.092658 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.093619 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.095575 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.096118 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.096217 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.100282 (+ 4065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:58559: BlockingRecv error: recv got EOF from 127.29.156.193:58559 (error 108)
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":390,"threads_started":1}
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:20 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:20.608024 10868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:20.598011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:50857)
0807 18:22:20.598798 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:20.598817 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:20.599103 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:20.599705 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:20.599711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:20.600209 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:20.600384 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.600393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.601690 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.601694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.602793 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:20.602807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.603692 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.603698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:20.603834 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.604500 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:20.604520 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:20.607763 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":472,"threads_started":1}
W20260807 18:22:20.608234 10867 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:20.598137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:50857 (local address 127.29.156.254:39059)
0807 18:22:20.598372 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:20.598382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:20.598925 (+ 543us) server_negotiation.cc:407] Connection header received
0807 18:22:20.599320 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:20.599325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:20.599427 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:20.599588 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:20.600530 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.601543 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.602982 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:20.603440 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:20.603625 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:20.607958 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:50857: BlockingRecv error: recv got EOF from 127.29.156.194:50857 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:21.108098 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:21.096560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:50255)
0807 18:22:21.097193 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:21.097223 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:21.097404 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:21.097791 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:21.097796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:21.098567 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:21.098762 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.098773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.100086 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.100090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.101597 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.101620 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.102808 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.102818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.103020 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.103927 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:21.103953 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:21.107645 (+ 3692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":333,"threads_started":1}
W20260807 18:22:21.108109 10867 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:21.096659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:50255 (local address 127.29.156.254:39059)
0807 18:22:21.096981 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:21.096989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:21.097326 (+ 337us) server_negotiation.cc:407] Connection header received
0807 18:22:21.097549 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:21.097562 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:21.097640 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:21.097755 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:21.098957 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.099848 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.101815 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.102594 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.102741 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.107857 (+ 5116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:50255: BlockingRecv error: recv got EOF from 127.29.156.195:50255 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:21.112251 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:21.101358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:50031)
0807 18:22:21.101959 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:21.101984 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:21.102147 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:21.102700 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:21.102705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:21.103223 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:21.103464 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.103475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.105059 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.105064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.106882 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.106899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.107785 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.107793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.108027 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.109059 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:21.109077 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:21.111980 (+ 2903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":296,"threads_started":1}
W20260807 18:22:21.112643 10876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:21.101455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:50031 (local address 127.29.156.254:39059)
0807 18:22:21.102306 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:21.102313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:21.102331 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:22:21.102381 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:21.102386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:21.102460 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:21.102582 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:21.103628 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.104893 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.107049 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.107606 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.107718 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.112226 (+ 4508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:50031: BlockingRecv error: recv got EOF from 127.29.156.193:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":588,"threads_started":1}
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:21 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:21.619565 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:21.609281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:60859)
0807 18:22:21.610001 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:21.610019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:21.610214 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:21.610626 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:21.610631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:21.611158 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:21.611356 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.611366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.612679 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.612692 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.613737 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:21.613750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.614552 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.614560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:21.614749 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.615433 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:21.615455 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:21.619138 (+ 3683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":422,"threads_started":1}
W20260807 18:22:21.619674 10876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:21.609373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:60859 (local address 127.29.156.254:39059)
0807 18:22:21.609658 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:21.609669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:21.610122 (+ 453us) server_negotiation.cc:407] Connection header received
0807 18:22:21.610384 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:21.610389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:21.610468 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:21.610581 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:21.611558 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.612490 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.613910 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:21.614358 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:21.614525 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:21.619382 (+ 4857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:60859: BlockingRecv error: recv got EOF from 127.29.156.194:60859 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:22.119760 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:22.109179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:58383)
0807 18:22:22.109806 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:22.109823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:22.109960 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:22.110341 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:22.110345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:22.110786 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:22.110974 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.110983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.112084 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.112088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.113288 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.113308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.114684 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.114691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.114936 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.115790 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:22.115819 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:22.119347 (+ 3528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":346,"threads_started":1}
W20260807 18:22:22.119889 10876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:22.109213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:58383 (local address 127.29.156.254:39059)
0807 18:22:22.109444 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:22.109449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:22.109901 (+ 452us) server_negotiation.cc:407] Connection header received
0807 18:22:22.110111 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:22.110115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:22.110181 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:22.110272 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:22.111139 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.111952 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.113502 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.114428 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.114589 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.119605 (+ 5016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:58383: BlockingRecv error: recv got EOF from 127.29.156.195:58383 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:22.123345 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:22.113341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:39619)
0807 18:22:22.114296 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:22.114319 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:22.114458 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:22.114942 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:22.114946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:22.115500 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:22.115745 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.115760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.116902 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.116908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.118205 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.118218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.119355 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.119360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.119605 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.120264 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:22.120290 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:22.123088 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":683,"threads_started":1}
W20260807 18:22:22.123636 10880 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:22.113440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:39619 (local address 127.29.156.254:39059)
0807 18:22:22.114616 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:22.114622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:22.114642 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:22:22.114692 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:22.114696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:22.114769 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:22.114914 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:22.115911 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.116738 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.118362 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.119160 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.119314 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.123301 (+ 3987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:39619: BlockingRecv error: recv got EOF from 127.29.156.193:39619 (error 108)
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":966,"threads_started":1}
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:22 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:22.629704 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:22.620776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:56755)
0807 18:22:22.621301 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:22.621317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:22.621489 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:22.621843 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:22.621848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:22.622324 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:22.622500 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.622511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.623696 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.623710 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.624639 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:22.624651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.625394 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.625401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:22.625574 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.626297 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:22.626316 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:22.629453 (+ 3137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":296,"threads_started":1}
W20260807 18:22:22.629864 10880 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:22.620833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:56755 (local address 127.29.156.254:39059)
0807 18:22:22.621019 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:22.621024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:22.621395 (+ 371us) server_negotiation.cc:407] Connection header received
0807 18:22:22.621616 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:22.621620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:22.621685 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:22.621811 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:22.622662 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.623462 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.624796 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:22.625233 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:22.625375 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:22.629653 (+ 4278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:56755: BlockingRecv error: recv got EOF from 127.29.156.194:56755 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:23.132421 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:23.121151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:58645)
0807 18:22:23.121800 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:23.121819 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:23.121999 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:23.122406 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:23.122411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:23.122853 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:23.123060 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.123070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.124680 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.124689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.126517 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.126533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.127607 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.127617 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.127841 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.128620 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:23.128639 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:23.132072 (+ 3433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":388,"threads_started":1}
W20260807 18:22:23.132556 10880 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:23.121404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:58645 (local address 127.29.156.254:39059)
0807 18:22:23.121689 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:23.121698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:23.121913 (+ 215us) server_negotiation.cc:407] Connection header received
0807 18:22:23.122156 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:23.122163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:23.122251 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:23.122370 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:23.123235 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.124479 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.126707 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.127326 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.127475 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.132306 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:58645: BlockingRecv error: recv got EOF from 127.29.156.195:58645 (error 108)
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":102}
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:23.133945 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:23.124432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:57563)
0807 18:22:23.124968 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:23.124991 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:23.125136 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:23.125675 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:23.125681 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:23.126258 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:23.126554 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.126569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.127913 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.127918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.129009 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.129022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.129813 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.129818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.129965 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.130660 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:23.130677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:23.133722 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":280,"threads_started":1}
W20260807 18:22:23.134299 10884 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:23.124527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:57563 (local address 127.29.156.254:39059)
0807 18:22:23.125287 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:23.125293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:23.125312 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:22:23.125380 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:23.125384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:23.125435 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:23.125640 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:23.126720 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.127775 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.129169 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.129613 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.129770 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.133907 (+ 4137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:57563: BlockingRecv error: recv got EOF from 127.29.156.193:57563 (error 108)
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":598,"thread_start_us":450,"threads_started":1}
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:23 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:23.644353 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:23.630837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:57379)
0807 18:22:23.631419 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:23.631436 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:23.631621 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:23.632059 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:23.632062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:23.632482 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:23.632663 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.632671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.634189 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.634194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.635258 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:23.635273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.636142 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.636147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:23.636293 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.641012 (+ 4719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:23.641037 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:23.644106 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":333,"threads_started":1}
W20260807 18:22:23.644809 10884 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:23.630917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:57379 (local address 127.29.156.254:39059)
0807 18:22:23.631110 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:23.631114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:23.631601 (+ 487us) server_negotiation.cc:407] Connection header received
0807 18:22:23.631750 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:23.631754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:23.631827 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:23.631979 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:23.632890 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.634031 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.635423 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:23.635973 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:23.636114 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:23.644367 (+ 8253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:57379: BlockingRecv error: recv got EOF from 127.29.156.194:57379 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:24.144266 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:24.133510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:33277)
0807 18:22:24.133999 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:24.134015 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:24.134140 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:24.134485 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:24.134490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:24.135045 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:24.135240 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.135250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.136704 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.136710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.138086 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.138104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.139174 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.139180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.139365 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.140324 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:24.140350 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:24.143921 (+ 3571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
W20260807 18:22:24.144312 10884 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:24.133611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33277 (local address 127.29.156.254:39059)
0807 18:22:24.133889 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:24.133904 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:24.134135 (+ 231us) server_negotiation.cc:407] Connection header received
0807 18:22:24.134232 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:24.134236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:24.134306 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:24.134393 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:24.135415 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.136541 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.138215 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.139006 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.139129 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.144122 (+ 4993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:33277: BlockingRecv error: recv got EOF from 127.29.156.195:33277 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:24.145838 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:24.134961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:43421)
0807 18:22:24.136124 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:24.136147 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:24.136267 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:24.136717 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:24.136722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:24.137284 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:24.137500 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.137513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.139035 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.139041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.140559 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.140576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.141375 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.141382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.141604 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.142267 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:24.142284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:24.145525 (+ 3241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":882,"threads_started":1}
W20260807 18:22:24.146199 10888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:24.135058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:43421 (local address 127.29.156.254:39059)
0807 18:22:24.135880 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:24.135886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:24.136242 (+ 356us) server_negotiation.cc:407] Connection header received
0807 18:22:24.136388 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:24.136393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:24.136478 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:24.136609 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:24.137720 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.138872 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.140725 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.141203 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.141302 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.145797 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:43421: BlockingRecv error: recv got EOF from 127.29.156.193:43421 (error 108)
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":544,"threads_started":1}
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:24 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:24.654511 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:24.645436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:35671)
0807 18:22:24.646020 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:24.646037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:24.646213 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:24.646658 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:24.646664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:24.647212 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:24.647426 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.647439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.648609 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.648613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.649601 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:24.649613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.650318 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.650323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:24.650455 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.651061 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:24.651078 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:24.654245 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":317,"threads_started":1}
W20260807 18:22:24.654703 10888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:24.645499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35671 (local address 127.29.156.254:39059)
0807 18:22:24.645698 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:24.645708 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:24.646162 (+ 454us) server_negotiation.cc:407] Connection header received
0807 18:22:24.646373 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:24.646379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:24.646470 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:24.646588 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:24.647620 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.648465 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.649745 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:24.650187 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:24.650286 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:24.654458 (+ 4172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:35671: BlockingRecv error: recv got EOF from 127.29.156.194:35671 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:25.156291 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:25.145384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:36493)
0807 18:22:25.146029 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:25.146054 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:25.146238 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:25.146669 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:25.146672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:25.147206 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:25.147367 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.147376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.149036 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.149041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.150325 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.150338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.151248 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.151254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.151450 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.152222 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:25.152243 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:25.155824 (+ 3581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":372,"threads_started":1}
W20260807 18:22:25.156344 10888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:25.145419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:36493 (local address 127.29.156.254:39059)
0807 18:22:25.145689 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:25.145694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:25.146176 (+ 482us) server_negotiation.cc:407] Connection header received
0807 18:22:25.146387 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:25.146392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:25.146503 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:25.146614 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:25.147648 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.148860 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.150486 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.151068 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.151168 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.156085 (+ 4917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:36493: BlockingRecv error: recv got EOF from 127.29.156.195:36493 (error 108)
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":100}
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:25.158099 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:25.146856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:44749)
0807 18:22:25.149216 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:25.149233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:25.149382 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:25.149784 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:25.149789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:25.150269 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:25.150456 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.150470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.151957 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.151963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.152912 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.152924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.153649 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.153657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.153836 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.154505 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:25.154523 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:25.157831 (+ 3308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1965,"thread_start_us":1830,"threads_started":1}
W20260807 18:22:25.158376 10892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:25.147012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:44749 (local address 127.29.156.254:39059)
0807 18:22:25.148481 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:25.148487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:25.149333 (+ 846us) server_negotiation.cc:407] Connection header received
0807 18:22:25.149520 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:25.149525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:25.149602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:25.149710 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:25.150640 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.151810 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.153072 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.153455 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.153592 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.158040 (+ 4448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:44749: BlockingRecv error: recv got EOF from 127.29.156.193:44749 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1355,"thread_start_us":1198,"threads_started":1}
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:25 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:25.664633 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:25.655665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:51161)
0807 18:22:25.656198 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:25.656216 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:25.656334 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:25.656721 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:25.656726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:25.657228 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:25.657396 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.657405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.658461 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.658468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.659423 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:25.659435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.660199 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.660206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:25.660373 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.661062 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:25.661096 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:25.664362 (+ 3266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":273,"threads_started":1}
W20260807 18:22:25.664835 10892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:25.655839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:51161 (local address 127.29.156.254:39059)
0807 18:22:25.656083 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:25.656089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:25.656296 (+ 207us) server_negotiation.cc:407] Connection header received
0807 18:22:25.656436 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:25.656440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:25.656528 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:25.656636 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:25.657550 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.658284 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.659583 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:25.660009 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:25.660147 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:25.664594 (+ 4447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:51161: BlockingRecv error: recv got EOF from 127.29.156.194:51161 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:26.168284 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:26.157428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:55957)
0807 18:22:26.158102 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:26.158124 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:26.158299 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:26.158690 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:26.158694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:26.159217 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:26.159386 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.159394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.160803 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.160807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.162117 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.162132 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.163043 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.163051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.163241 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.164006 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:26.164032 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:26.167876 (+ 3844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:22:26.168481 10892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:26.157565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:55957 (local address 127.29.156.254:39059)
0807 18:22:26.157851 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:26.157859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:26.158228 (+ 369us) server_negotiation.cc:407] Connection header received
0807 18:22:26.158436 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:26.158467 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:26.158534 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:26.158646 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:26.159565 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.160657 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.162257 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.162852 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.162984 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.168130 (+ 5146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:55957: BlockingRecv error: recv got EOF from 127.29.156.195:55957 (error 108)
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":98}
I20260807 18:22:26.169588 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:26.159091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:57773)
0807 18:22:26.159957 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:26.159977 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:26.160100 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:26.160633 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:26.160637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:26.161068 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:26.161289 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.161301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.162779 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.162785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.163875 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.163888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.164704 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.164713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.164956 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.165671 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:26.165691 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:26.169334 (+ 3643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":604,"threads_started":1}
W20260807 18:22:26.169915 10905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:26.159167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:57773 (local address 127.29.156.254:39059)
0807 18:22:26.160368 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:26.160372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:26.160387 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:22:26.160422 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:26.160425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:26.160472 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:26.160564 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:26.161466 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.162616 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.164098 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.164520 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.164612 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.169595 (+ 4983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:57773: BlockingRecv error: recv got EOF from 127.29.156.193:57773 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1094,"thread_start_us":864,"threads_started":1}
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:26 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:26.676713 10907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:26.665744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:38127)
0807 18:22:26.666337 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:26.666367 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:26.666523 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:26.666896 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:26.666902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:26.667343 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:26.667598 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.667608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.669605 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.669614 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.671045 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:26.671060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.671943 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.671951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:26.672172 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.672895 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:26.672916 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:26.676310 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":331,"threads_started":1}
W20260807 18:22:26.676811 10905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:26.665851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:38127 (local address 127.29.156.254:39059)
0807 18:22:26.666058 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:26.666063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:26.666518 (+ 455us) server_negotiation.cc:407] Connection header received
0807 18:22:26.666653 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:26.666657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:26.666731 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:26.666827 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:26.667786 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.669383 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.671223 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:26.671769 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:26.671923 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:26.676563 (+ 4640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:38127: BlockingRecv error: recv got EOF from 127.29.156.194:38127 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.195@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:27.179661 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:27.169445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.195:41293)
0807 18:22:27.170124 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:27.170144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:27.170324 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:27.170735 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:27.170742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:27.171313 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:27.171632 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.171650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.173250 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.173258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.174308 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.174321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.175209 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.175215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.175398 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.176171 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:27.176191 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:27.179326 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":324,"threads_started":1}
W20260807 18:22:27.179993 10905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:27.169676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:41293 (local address 127.29.156.254:39059)
0807 18:22:27.169930 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:27.169937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:27.170250 (+ 313us) server_negotiation.cc:407] Connection header received
0807 18:22:27.170466 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:27.170471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:27.170566 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:27.170690 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:27.171849 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.173090 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.174472 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.175040 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.175190 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.179644 (+ 4454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.195:41293: BlockingRecv error: recv got EOF from 127.29.156.195:41293 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.193@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:27.181097 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:27.170779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.193:40321)
0807 18:22:27.172675 (+ 1896us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:27.172700 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:27.172855 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:27.173233 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:27.173237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:27.173640 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:27.173799 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.173808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.174914 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.174919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.175913 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.175926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.176641 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.176648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.176831 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.177445 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:27.177465 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:27.180820 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1761,"thread_start_us":1631,"threads_started":1}
W20260807 18:22:27.181373 10912 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:27.171034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40321 (local address 127.29.156.254:39059)
0807 18:22:27.172094 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:27.172100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:27.172795 (+ 695us) server_negotiation.cc:407] Connection header received
0807 18:22:27.173012 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:27.173017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:27.173083 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:27.173194 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:27.173957 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.174755 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.176058 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.176479 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.176611 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.181072 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.193:40321: BlockingRecv error: recv got EOF from 127.29.156.193:40321 (error 108)
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":778,"threads_started":1}
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:22:27 dist-test-slave-sb2z krb5kdc[9434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.156.194@KRBTEST.COM for oryx/127.29.156.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:22:27.687934 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:27.678244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:39059 (local address 127.29.156.194:38291)
0807 18:22:27.678859 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:27.678875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:27.679052 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:27.679434 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:27.679439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:27.679919 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:27.680172 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.680184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.681482 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.681487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.682822 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:27.682836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.683672 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.683679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:27.683852 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.684499 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:27.684518 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:27.687667 (+ 3149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.156.254:39059: Server oryx/127.29.156.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":318,"threads_started":1}
W20260807 18:22:27.688089 10912 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:27.678399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:38291 (local address 127.29.156.254:39059)
0807 18:22:27.678674 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:27.678684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:27.678976 (+ 292us) server_negotiation.cc:407] Connection header received
0807 18:22:27.679192 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:27.679197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:27.679275 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:27.679384 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:27.680392 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.681215 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.682993 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.683453 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.683669 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.687874 (+ 4205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.156.194:38291: BlockingRecv error: recv got EOF from 127.29.156.194:38291 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:22:27.982906 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10275
I20260807 18:22:27.997196 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:27.984175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.29.156.193:39035)
0807 18:22:27.985072 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:27.985082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:27.985123 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:22:27.985224 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:27.985236 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:27.985623 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:27.985916 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:27.987308 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.988655 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.989881 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:27.990497 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:27.990613 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:27.993372 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:27.993381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:27.993396 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:27.993437 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:27.996668 (+ 3231us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:22:27.996953 (+ 285us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50016: Request ticket server kudu/127.29.156.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":392,"threads_started":1}
W20260807 18:22:27.997375 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10275: Not authorized: Client connection negotiation failed: client connection to 127.29.156.193:39035: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:22:27.997646 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 10275
I20260807 18:22:28.029268 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10473
I20260807 18:22:28.043118 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:28.030595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.29.156.194:45261)
0807 18:22:28.031254 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:28.031261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:28.031288 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:22:28.031348 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:28.031355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:28.031649 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:28.031816 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:28.033296 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.034671 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.036117 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.036705 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.036865 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:28.039625 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:28.039637 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:28.039653 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:28.039695 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:28.042347 (+ 2652us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:22:28.042724 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48304: Request ticket server kudu/127.29.156.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":299,"threads_started":1}
W20260807 18:22:28.043167 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10473: Not authorized: Client connection negotiation failed: client connection to 127.29.156.194:45261: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:22:28.043249 10916 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48304: Request ticket server kudu/127.29.156.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:22:28.043321 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 10473
I20260807 18:22:28.068269 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10647
I20260807 18:22:28.081071 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:28.069325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.29.156.195:42417)
0807 18:22:28.069945 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:28.069952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:28.069982 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:22:28.070046 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:28.070052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:28.070288 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:28.070478 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:28.072038 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.073093 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.074376 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.074947 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.075063 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:28.077874 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:28.077883 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:28.077898 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:28.077943 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:28.080540 (+ 2597us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:22:28.080808 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46066: Request ticket server kudu/127.29.156.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":324,"threads_started":1}
W20260807 18:22:28.081205 10917 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46066: Request ticket server kudu/127.29.156.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:22:28.081264 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10647: Not authorized: Client connection negotiation failed: client connection to 127.29.156.195:42417: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.156.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:22:28.081411 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 10647
I20260807 18:22:28.108417 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10183
I20260807 18:22:28.109984 10265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:28.238523 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 10183
2026-08-07T18:22:28Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21971 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:22:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:28Z Disabled control of system clock
I20260807 18:22:28.340687 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41717
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:22:28.687896 10924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:28.688217 10924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:28.688300 10924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:28.696847 10924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:28.696988 10924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:28.697032 10924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:28.697101 10924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:28.697152 10924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:28.707947 10924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:28.709388 10924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:28.711273 10924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:28.725709 10930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:28.725740 10929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:28.725665 10932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:28.726248 10924 server_base.cc:1034] running on GCE node
I20260807 18:22:28.727404 10924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:28.729010 10924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:28.730268 10924 hybrid_clock.cc:648] HybridClock initialized: now 1786126948730207 us; error 32 us; skew 500 ppm
I20260807 18:22:28.730657 10924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:28.737071 10924 webserver.cc:460] Webserver started at http://127.29.156.254:38267/ using document root <none> and password file <none>
I20260807 18:22:28.737854 10924 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:28.737964 10924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:28.738291 10924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:28.741765 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "f7551b4d43294c4d8d6f3194c376c72a"
format_stamp: "Formatted at 2026-08-07 18:22:28 on dist-test-slave-sb2z"
server_key: "c6faded0a3eeecc0d7868c7b85cd1692"
server_key_iv: "23168bb5b251471c2b935fc665ef5c0f"
server_key_version: "encryptionkey@0"
I20260807 18:22:28.742674 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "f7551b4d43294c4d8d6f3194c376c72a"
format_stamp: "Formatted at 2026-08-07 18:22:28 on dist-test-slave-sb2z"
server_key: "c6faded0a3eeecc0d7868c7b85cd1692"
server_key_iv: "23168bb5b251471c2b935fc665ef5c0f"
server_key_version: "encryptionkey@0"
I20260807 18:22:28.748715 10924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:22:28.752513 10940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:28.753952 10924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:22:28.754215 10924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "f7551b4d43294c4d8d6f3194c376c72a"
format_stamp: "Formatted at 2026-08-07 18:22:28 on dist-test-slave-sb2z"
server_key: "c6faded0a3eeecc0d7868c7b85cd1692"
server_key_iv: "23168bb5b251471c2b935fc665ef5c0f"
server_key_version: "encryptionkey@0"
I20260807 18:22:28.754473 10924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:28.778287 10924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:28.779277 10924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:28.779635 10924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:28.799890 10924 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:41717
I20260807 18:22:28.799935 11014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:41717 every 8 connection(s)
I20260807 18:22:28.801362 10924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:28.805346 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:28.810359 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 10924
I20260807 18:22:28.810568 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:22:28.811465 30323 external_mini_cluster.cc:1428] Setting key ecd0f4fa89c4c6eafdaca651afe73cb8
I20260807 18:22:28.817610 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a: Bootstrap starting.
I20260807 18:22:28.820806 11015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:28.821862 11015 log.cc:824] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:28.824918 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a: No bootstrap required, opened a new log
I20260807 18:22:28.826736 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:28.813907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.29.156.254:41717)
0807 18:22:28.815072 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:28.815089 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:28.815141 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:22:28.817072 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:28.817097 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:28.817574 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:28.818046 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:28.818675 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.820193 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.821750 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:28.822551 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:28.822788 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:28.824150 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:28.824160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:28.824175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:28.824785 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:28.824879 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:28.824918 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:28.825088 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:28.825347 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:22:28.825689 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":356,"threads_started":1}
I20260807 18:22:28.833472 11015 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } }
I20260807 18:22:28.833877 11015 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:28.833988 11015 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7551b4d43294c4d8d6f3194c376c72a, State: Initialized, Role: FOLLOWER
I20260807 18:22:28.834625 11015 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } }
I20260807 18:22:28.834872 11015 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:28.835024 11015 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:28.835218 11015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:28.837795 11015 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } }
I20260807 18:22:28.838318 11015 leader_election.cc:302] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7551b4d43294c4d8d6f3194c376c72a; no voters:
I20260807 18:22:28.838987 11015 leader_election.cc:288] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:22:28.839262 11020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:28.840466 11020 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [term 1 LEADER]: Becoming Leader. State: Replica: f7551b4d43294c4d8d6f3194c376c72a, State: Running, Role: LEADER
I20260807 18:22:28.841082 11020 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } }
I20260807 18:22:28.841710 11015 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:28.844797 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7551b4d43294c4d8d6f3194c376c72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } } }
I20260807 18:22:28.844803 11022 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7551b4d43294c4d8d6f3194c376c72a. Latest consensus state: current_term: 1 leader_uuid: "f7551b4d43294c4d8d6f3194c376c72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7551b4d43294c4d8d6f3194c376c72a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41717 } } }
I20260807 18:22:28.845346 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:28.845402 11022 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:28.852563 11028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:28.858337 11028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:28.867048 11028 catalog_manager.cc:1347] Generated new cluster ID: f714b506f0e24463a44202e8e2201ef4
I20260807 18:22:28.867178 11028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:28.891795 11028 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:22:28.893270 11028 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:28.903669 11028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7551b4d43294c4d8d6f3194c376c72a: Generated new TSK 0
I20260807 18:22:28.904793 11028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:22:28.919065 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--builtin_ntp_servers=127.29.156.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:22:29.284236 11043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:29.284541 11043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:29.284628 11043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:29.291997 11043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:29.292125 11043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:29.292232 11043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:29.301544 11043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:29.302879 11043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:29.304800 11043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:29.318804 11051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:29.318686 11049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:29.318619 11048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:29.321614 11043 server_base.cc:1034] running on GCE node
I20260807 18:22:29.322487 11043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:29.323825 11043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:29.325119 11043 hybrid_clock.cc:648] HybridClock initialized: now 1786126949325011 us; error 67 us; skew 500 ppm
I20260807 18:22:29.325518 11043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:29.328451 11043 webserver.cc:460] Webserver started at http://127.29.156.193:37487/ using document root <none> and password file <none>
I20260807 18:22:29.329164 11043 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:29.329252 11043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:29.329524 11043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:29.332541 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "4e7c5c7cd51340f2903b424de7db7f9b"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "6205d622a36a217b3dff377191872a6a"
server_key_iv: "e81bbe3adc2f3a641379b8b028adcd84"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.333324 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "4e7c5c7cd51340f2903b424de7db7f9b"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "6205d622a36a217b3dff377191872a6a"
server_key_iv: "e81bbe3adc2f3a641379b8b028adcd84"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.338609 11043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:22:29.342253 11058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.343883 11043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:22:29.344082 11043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "4e7c5c7cd51340f2903b424de7db7f9b"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "6205d622a36a217b3dff377191872a6a"
server_key_iv: "e81bbe3adc2f3a641379b8b028adcd84"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.344311 11043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:29.365293 11043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:29.366214 11043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:29.366544 11043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:29.367605 11043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:29.369119 11043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:29.369238 11043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.369347 11043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:29.369443 11043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.406040 11043 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:32889
I20260807 18:22:29.406148 11207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:32889 every 8 connection(s)
I20260807 18:22:29.407466 11043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:29.411876 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11043
I20260807 18:22:29.412093 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:22:29.413017 30323 external_mini_cluster.cc:1428] Setting key 482ffc0889400b5117d51d5bbbad0040
I20260807 18:22:29.416394 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--builtin_ntp_servers=127.29.156.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:22:29.422773 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:29.410261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:47885 (local address 127.29.156.254:41717)
0807 18:22:29.410713 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:29.410718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:29.412373 (+ 1655us) server_negotiation.cc:407] Connection header received
0807 18:22:29.414750 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:29.414755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:29.414826 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:29.415052 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:29.416266 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.417513 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.419181 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.420015 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.420175 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:29.421694 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:29.421703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:29.421711 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:29.421740 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:29.421797 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:29.421817 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:29.421921 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:29.422163 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:22:29.422287 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260807 18:22:29.423753 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:29.410850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41717 (local address 127.29.156.193:47885)
0807 18:22:29.412161 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:29.412215 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:29.414436 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:29.415280 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:29.415294 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:29.415396 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:29.416075 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:29.416093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.417685 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.417689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:29.418990 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:29.419002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.420262 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.420267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:29.420514 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:29.421331 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:29.421334 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:29.421377 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:29.421465 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:29.421928 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:29.421938 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:29.421945 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:22:29.422117 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:22:29.422437 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":313,"threads_started":1}
I20260807 18:22:29.425522 11208 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41717
I20260807 18:22:29.425879 11208 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:29.426951 11208 heartbeater.cc:511] Master 127.29.156.254:41717 requested a full tablet report, sending...
I20260807 18:22:29.429383 10964 ts_manager.cc:194] Registered new tserver with Master: 4e7c5c7cd51340f2903b424de7db7f9b (127.29.156.193:32889)
I20260807 18:22:29.430896 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.193:47885
W20260807 18:22:29.774893 11213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:29.775206 11213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:29.775295 11213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:29.782814 11213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:29.782940 11213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:29.783064 11213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:29.792538 11213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:29.793960 11213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:29.795956 11213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:29.810935 11223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:29.810979 11226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:29.814492 11222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:29.816572 11213 server_base.cc:1034] running on GCE node
I20260807 18:22:29.817304 11213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:29.818418 11213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:29.819659 11213 hybrid_clock.cc:648] HybridClock initialized: now 1786126949819598 us; error 41 us; skew 500 ppm
I20260807 18:22:29.820031 11213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:29.822587 11213 webserver.cc:460] Webserver started at http://127.29.156.194:40401/ using document root <none> and password file <none>
I20260807 18:22:29.823371 11213 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:29.823568 11213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:29.823943 11213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:29.826992 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "6e11c373e5da4a5c9159c0a1997a96d8"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "d7cd974ce6d36ca67bc1b1a356e1d7cc"
server_key_iv: "e550bbada324efc37868113bddd48b02"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.828015 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "6e11c373e5da4a5c9159c0a1997a96d8"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "d7cd974ce6d36ca67bc1b1a356e1d7cc"
server_key_iv: "e550bbada324efc37868113bddd48b02"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.833830 11213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:22:29.837662 11234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.839115 11213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:22:29.839320 11213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "6e11c373e5da4a5c9159c0a1997a96d8"
format_stamp: "Formatted at 2026-08-07 18:22:29 on dist-test-slave-sb2z"
server_key: "d7cd974ce6d36ca67bc1b1a356e1d7cc"
server_key_iv: "e550bbada324efc37868113bddd48b02"
server_key_version: "encryptionkey@0"
I20260807 18:22:29.839663 11213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:29.857766 11213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:29.858793 11213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:29.859136 11213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:29.860424 11213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:29.861923 11213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:29.862015 11213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.862102 11213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:29.862157 11213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:29.899020 11213 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:35707
I20260807 18:22:29.899122 11380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:35707 every 8 connection(s)
I20260807 18:22:29.900442 11213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:29.901367 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11213
I20260807 18:22:29.901566 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:22:29.902232 30323 external_mini_cluster.cc:1428] Setting key fde7bd66ccf9468c51eb9b897ccbfde6
I20260807 18:22:29.905496 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--builtin_ntp_servers=127.29.156.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:22:29.915547 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:29.903719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:43321 (local address 127.29.156.254:41717)
0807 18:22:29.903952 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:29.903959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:29.905847 (+ 1888us) server_negotiation.cc:407] Connection header received
0807 18:22:29.907869 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:29.907877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:29.907969 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:29.908108 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:29.909240 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.910357 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.912201 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.912896 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.913043 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:29.914557 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:29.914565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:29.914573 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:29.914599 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:29.914642 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:29.914661 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:29.914756 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:29.914980 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 18:22:29.915098 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:22:29.916632 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:29.904460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41717 (local address 127.29.156.194:43321)
0807 18:22:29.905676 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:29.905726 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:29.907579 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:29.908325 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:29.908334 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:29.908423 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:29.909057 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:29.909076 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.910543 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.910548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:29.912051 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:29.912065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:29.913137 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:29.913143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:29.913377 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:29.914205 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:29.914208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:29.914252 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:29.914336 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:29.914767 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:29.914776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:29.914782 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:22:29.914941 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 18:22:29.915262 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":411,"threads_started":1}
I20260807 18:22:29.918349 11383 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41717
I20260807 18:22:29.918665 11383 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:29.919510 11383 heartbeater.cc:511] Master 127.29.156.254:41717 requested a full tablet report, sending...
I20260807 18:22:29.921229 10964 ts_manager.cc:194] Registered new tserver with Master: 6e11c373e5da4a5c9159c0a1997a96d8 (127.29.156.194:35707)
I20260807 18:22:29.922031 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.194:43321
W20260807 18:22:30.277962 11386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:30.278353 11386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:30.278542 11386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:30.286109 11386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:30.286266 11386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:30.286448 11386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:30.295818 11386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:30.297227 11386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:30.299046 11386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:30.312702 11392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:30.312650 11395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:30.312726 11393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:30.313344 11386 server_base.cc:1034] running on GCE node
I20260807 18:22:30.314023 11386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:30.315007 11386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:30.316241 11386 hybrid_clock.cc:648] HybridClock initialized: now 1786126950316189 us; error 47 us; skew 500 ppm
I20260807 18:22:30.316663 11386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:30.318981 11386 webserver.cc:460] Webserver started at http://127.29.156.195:40125/ using document root <none> and password file <none>
I20260807 18:22:30.319820 11386 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:30.319957 11386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:30.320289 11386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:30.323182 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "cc620349de964848a7d69854a5bae4ac"
format_stamp: "Formatted at 2026-08-07 18:22:30 on dist-test-slave-sb2z"
server_key: "f38d81636e75a4b87c2cce11cf00f058"
server_key_iv: "cc153f98174b3f96d4a823a729f9fb71"
server_key_version: "encryptionkey@0"
I20260807 18:22:30.324019 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "cc620349de964848a7d69854a5bae4ac"
format_stamp: "Formatted at 2026-08-07 18:22:30 on dist-test-slave-sb2z"
server_key: "f38d81636e75a4b87c2cce11cf00f058"
server_key_iv: "cc153f98174b3f96d4a823a729f9fb71"
server_key_version: "encryptionkey@0"
I20260807 18:22:30.329180 11386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:22:30.332579 11405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:30.334002 11386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:22:30.334185 11386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "cc620349de964848a7d69854a5bae4ac"
format_stamp: "Formatted at 2026-08-07 18:22:30 on dist-test-slave-sb2z"
server_key: "f38d81636e75a4b87c2cce11cf00f058"
server_key_iv: "cc153f98174b3f96d4a823a729f9fb71"
server_key_version: "encryptionkey@0"
I20260807 18:22:30.334411 11386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:30.351297 11386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:30.352329 11386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:30.352676 11386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:30.353754 11386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:30.355252 11386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:30.355382 11386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:30.355556 11386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:30.355649 11386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:22:30.391778 11386 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:37593
I20260807 18:22:30.391831 11558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:37593 every 8 connection(s)
I20260807 18:22:30.393112 11386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:30.400575 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11386
I20260807 18:22:30.400770 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:22:30.401316 30323 external_mini_cluster.cc:1428] Setting key d9a7ab49445f8e925606e43be52ada72
I20260807 18:22:30.406701 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:30.395834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43941 (local address 127.29.156.254:41717)
0807 18:22:30.396081 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:30.396086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:30.397379 (+ 1293us) server_negotiation.cc:407] Connection header received
0807 18:22:30.399210 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:30.399215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:30.399304 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:30.399432 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:30.400534 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.401452 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.403481 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.404009 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.404153 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:30.405795 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:30.405801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:30.405806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:30.405827 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:30.405867 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:30.405882 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:30.405990 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:30.406216 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:22:30.406320 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:22:30.407684 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:30.396309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:41717 (local address 127.29.156.195:43941)
0807 18:22:30.397207 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:30.397243 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:30.398909 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:30.399703 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:30.399711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:30.399795 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:30.400358 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:30.400373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.401629 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.401634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:30.403327 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:30.403345 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.404246 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.404256 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:30.404526 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:30.405372 (+ 846us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:30.405378 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:30.405461 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:30.405582 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:30.405993 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:30.406002 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:30.406008 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:22:30.406158 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 18:22:30.406457 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":286,"threads_started":1}
I20260807 18:22:30.409307 11559 heartbeater.cc:347] Connected to a master server at 127.29.156.254:41717
I20260807 18:22:30.409647 11559 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:30.410561 11559 heartbeater.cc:511] Master 127.29.156.254:41717 requested a full tablet report, sending...
I20260807 18:22:30.412055 10964 ts_manager.cc:194] Registered new tserver with Master: cc620349de964848a7d69854a5bae4ac (127.29.156.195:37593)
I20260807 18:22:30.412820 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.195:43941
I20260807 18:22:30.417161 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:22:30.424131 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:30.422320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.29.156.254:41717)
0807 18:22:30.422668 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:30.422675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:30.422956 (+ 281us) server_negotiation.cc:407] Connection header received
0807 18:22:30.423166 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:30.423171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:30.423241 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:30.423337 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:30.423888 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59166: BlockingRecv error: recv got EOF from 127.0.0.1:59166 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
I20260807 18:22:30.424759 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11043
I20260807 18:22:30.432905 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:30.425834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.29.156.193:32889)
0807 18:22:30.426408 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:30.426430 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:30.426456 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:22:30.426511 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:30.426518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:30.426725 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:30.426850 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:30.427306 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.428390 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.429761 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.430317 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.430424 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:30.431589 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:30.431598 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:30.431611 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:30.432192 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:30.432236 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:30.432257 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:30.432347 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:30.432499 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:22:30.432609 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":301,"threads_started":1}
I20260807 18:22:30.434079 11202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:30.434309 11208 heartbeater.cc:503] Master 127.29.156.254:41717 was elected leader, sending a full tablet report...
I20260807 18:22:30.519889 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11043
W20260807 18:22:30.545722 10944 connection.cc:535] server connection from 127.29.156.193:47885 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:22:30.546139 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11213
I20260807 18:22:30.555264 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:30.547131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.29.156.194:35707)
0807 18:22:30.547968 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:30.547976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:30.548008 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:22:30.548085 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:30.548092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:30.548339 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:30.548495 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:30.548843 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.550032 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.551631 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.552336 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.552457 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:30.553619 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:30.553629 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:30.553643 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:30.554334 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:30.554389 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:30.554410 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:30.554508 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:30.554662 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:22:30.554838 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":453,"threads_started":1}
I20260807 18:22:30.556643 11374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:30.640806 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11213
I20260807 18:22:30.665516 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11386
I20260807 18:22:30.674288 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:30.666572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.29.156.195:37593)
0807 18:22:30.667292 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:30.667307 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:30.667338 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:22:30.667401 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:30.667407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:30.667702 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:30.667884 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:30.668381 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.669500 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.670817 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:30.671401 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:30.671578 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:30.672722 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:30.672730 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:30.672744 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:30.673399 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:30.673452 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:30.673472 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:30.673610 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:30.673762 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:22:30.673921 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":345,"threads_started":1}
I20260807 18:22:30.675670 11553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:30.762473 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11386
I20260807 18:22:30.788594 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 10924
I20260807 18:22:30.790110 11005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:30.899041 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 10924
2026-08-07T18:22:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2653 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:22:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:30Z Disabled control of system clock
I20260807 18:22:30.999318 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:34595
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:34595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:22:31.356101 11582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:31.356415 11582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:31.356597 11582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:31.364271 11582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:31.364388 11582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:31.364485 11582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:31.364589 11582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:31.364660 11582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:31.374044 11582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:34595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:34595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:31.375465 11582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:31.377384 11582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:31.389832 11587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:31.389855 11591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:31.389894 11593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:31.390628 11582 server_base.cc:1034] running on GCE node
I20260807 18:22:31.391445 11582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:31.392935 11582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:31.394183 11582 hybrid_clock.cc:648] HybridClock initialized: now 1786126951394149 us; error 54 us; skew 500 ppm
I20260807 18:22:31.394551 11582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:31.397092 11582 webserver.cc:460] Webserver started at http://127.29.156.254:46389/ using document root <none> and password file <none>
I20260807 18:22:31.397817 11582 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:31.397966 11582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:31.398307 11582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:31.401424 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "18c659d6132f4385a07c82eb46ce645a"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "750d1828d8af07d83c22e746c8b08d6b"
server_key_iv: "c1e4263c75d396188c904b46426c1fee"
server_key_version: "encryptionkey@0"
I20260807 18:22:31.402280 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "18c659d6132f4385a07c82eb46ce645a"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "750d1828d8af07d83c22e746c8b08d6b"
server_key_iv: "c1e4263c75d396188c904b46426c1fee"
server_key_version: "encryptionkey@0"
I20260807 18:22:31.407696 11582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:22:31.411712 11600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:31.413363 11582 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:22:31.413600 11582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "18c659d6132f4385a07c82eb46ce645a"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "750d1828d8af07d83c22e746c8b08d6b"
server_key_iv: "c1e4263c75d396188c904b46426c1fee"
server_key_version: "encryptionkey@0"
I20260807 18:22:31.413869 11582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:31.420360 11582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:31.421233 11582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:31.421557 11582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:31.442061 11582 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:34595
I20260807 18:22:31.442127 11674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:34595 every 8 connection(s)
I20260807 18:22:31.443521 11582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:31.447770 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:31.451848 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11582
I20260807 18:22:31.452104 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:22:31.452786 30323 external_mini_cluster.cc:1428] Setting key 5f273202f2852df21608cd6ce29aa741
I20260807 18:22:31.460614 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a: Bootstrap starting.
I20260807 18:22:31.462713 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:31.455175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.29.156.254:34595)
0807 18:22:31.456336 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:31.456348 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:31.456394 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:22:31.458783 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:31.458808 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:31.459265 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:31.459700 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:31.460994 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:31.461001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:31.461015 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:31.461031 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:31.461095 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:31.461112 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:31.461306 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:31.461418 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 18:22:31.461730 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":466,"threads_started":1}
I20260807 18:22:31.464692 11675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:31.465972 11675 log.cc:824] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:31.469316 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a: No bootstrap required, opened a new log
I20260807 18:22:31.479243 11675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } }
I20260807 18:22:31.479841 11675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:31.479943 11675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c659d6132f4385a07c82eb46ce645a, State: Initialized, Role: FOLLOWER
I20260807 18:22:31.480665 11675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } }
I20260807 18:22:31.480968 11675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:31.481105 11675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:31.481273 11675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:31.485033 11675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } }
I20260807 18:22:31.485884 11675 leader_election.cc:302] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18c659d6132f4385a07c82eb46ce645a; no voters:
I20260807 18:22:31.486604 11675 leader_election.cc:288] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:22:31.486759 11684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:31.487713 11684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [term 1 LEADER]: Becoming Leader. State: Replica: 18c659d6132f4385a07c82eb46ce645a, State: Running, Role: LEADER
I20260807 18:22:31.488327 11684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } }
I20260807 18:22:31.489248 11675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:31.491457 11685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18c659d6132f4385a07c82eb46ce645a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } } }
I20260807 18:22:31.491925 11685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:31.491729 11686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18c659d6132f4385a07c82eb46ce645a. Latest consensus state: current_term: 1 leader_uuid: "18c659d6132f4385a07c82eb46ce645a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c659d6132f4385a07c82eb46ce645a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 34595 } } }
I20260807 18:22:31.492329 11686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:31.498695 11692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:31.504247 11692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:31.514379 11692 catalog_manager.cc:1347] Generated new cluster ID: 9be7483b07e943bd851aae24c2eee598
I20260807 18:22:31.514550 11692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:31.533814 11692 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:22:31.534323 11692 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:31.545709 11692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18c659d6132f4385a07c82eb46ce645a: Generated new TSK 0
I20260807 18:22:31.546665 11692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:22:31.558390 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--builtin_ntp_servers=127.29.156.212:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:22:31.940793 11711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:31.941154 11711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:31.941321 11711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:31.948925 11711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:31.949055 11711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:31.949267 11711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:31.958662 11711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:31.960099 11711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:31.962126 11711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:31.976540 11718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:31.976569 11719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:31.976562 11721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:31.977661 11711 server_base.cc:1034] running on GCE node
I20260807 18:22:31.978353 11711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:31.979362 11711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:31.980588 11711 hybrid_clock.cc:648] HybridClock initialized: now 1786126951980520 us; error 83 us; skew 500 ppm
I20260807 18:22:31.980967 11711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:31.983502 11711 webserver.cc:460] Webserver started at http://127.29.156.193:33699/ using document root <none> and password file <none>
I20260807 18:22:31.984298 11711 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:31.984414 11711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:31.984762 11711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:31.987905 11711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "f81de501c3a44a5ea3edfb4d45a08b1e"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "0e0fca5d589d645fd4712de59bc9e6cc"
server_key_iv: "f9534c88460397734055ff691b6ee65b"
server_key_version: "encryptionkey@0"
I20260807 18:22:31.988757 11711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "f81de501c3a44a5ea3edfb4d45a08b1e"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "0e0fca5d589d645fd4712de59bc9e6cc"
server_key_iv: "f9534c88460397734055ff691b6ee65b"
server_key_version: "encryptionkey@0"
I20260807 18:22:31.994464 11711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:22:31.998188 11731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:31.999825 11711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:22:32.000012 11711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "f81de501c3a44a5ea3edfb4d45a08b1e"
format_stamp: "Formatted at 2026-08-07 18:22:31 on dist-test-slave-sb2z"
server_key: "0e0fca5d589d645fd4712de59bc9e6cc"
server_key_iv: "f9534c88460397734055ff691b6ee65b"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.000249 11711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:32.006358 11711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:32.007246 11711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:32.007578 11711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:32.008594 11711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:32.010090 11711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:32.010226 11711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.010353 11711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:32.010453 11711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.045812 11711 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:46259
I20260807 18:22:32.045853 11883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:46259 every 8 connection(s)
I20260807 18:22:32.047256 11711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:32.053898 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11711
I20260807 18:22:32.054203 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:22:32.054903 30323 external_mini_cluster.cc:1428] Setting key 2425e07772b74e75fe5b07cfb1e3cce6
I20260807 18:22:32.056721 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:32.050239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38271 (local address 127.29.156.254:34595)
0807 18:22:32.050702 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:32.050707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:32.052135 (+ 1428us) server_negotiation.cc:407] Connection header received
0807 18:22:32.053997 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:32.054003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:32.054091 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:32.054229 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:32.055801 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:32.055808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:32.055819 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:32.055837 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:32.055900 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:32.055912 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:32.056057 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:32.056319 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 18:22:32.056392 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
I20260807 18:22:32.057955 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:32.050867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:34595 (local address 127.29.156.193:38271)
0807 18:22:32.051958 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:32.052003 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:32.053687 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:32.054495 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:32.054504 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:32.054606 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:32.055413 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:32.055416 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:32.055454 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:32.055608 (+ 154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:32.056019 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:32.056033 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:32.056042 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:22:32.056408 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 18:22:32.056679 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":303,"threads_started":1}
I20260807 18:22:32.058143 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--builtin_ntp_servers=127.29.156.212:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:22:32.059808 11884 heartbeater.cc:347] Connected to a master server at 127.29.156.254:34595
I20260807 18:22:32.060175 11884 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:32.060913 11884 heartbeater.cc:511] Master 127.29.156.254:34595 requested a full tablet report, sending...
I20260807 18:22:32.063386 11627 ts_manager.cc:194] Registered new tserver with Master: f81de501c3a44a5ea3edfb4d45a08b1e (127.29.156.193:46259)
W20260807 18:22:32.408274 11888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:32.408629 11888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:32.408809 11888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:32.416304 11888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:32.416457 11888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:32.416651 11888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:32.426668 11888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:32.428104 11888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:32.429929 11888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:32.446065 11894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:32.446066 11896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:32.447613 11888 server_base.cc:1034] running on GCE node
W20260807 18:22:32.451823 11899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:32.452749 11888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:32.454006 11888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:32.455260 11888 hybrid_clock.cc:648] HybridClock initialized: now 1786126952455212 us; error 42 us; skew 500 ppm
I20260807 18:22:32.455684 11888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:32.458659 11888 webserver.cc:460] Webserver started at http://127.29.156.194:40861/ using document root <none> and password file <none>
I20260807 18:22:32.459573 11888 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:32.459707 11888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:32.460085 11888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:32.463506 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "eb8b2d71c99141a8b035fdad8e81d5ae"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "07ccd7bb08f36b6e6f7bfb3a720749a3"
server_key_iv: "ef31dc9f1afb83e019392d8e42e95a79"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.464455 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "eb8b2d71c99141a8b035fdad8e81d5ae"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "07ccd7bb08f36b6e6f7bfb3a720749a3"
server_key_iv: "ef31dc9f1afb83e019392d8e42e95a79"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.469916 11888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 18:22:32.473601 11907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.475003 11888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:22:32.475222 11888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "eb8b2d71c99141a8b035fdad8e81d5ae"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "07ccd7bb08f36b6e6f7bfb3a720749a3"
server_key_iv: "ef31dc9f1afb83e019392d8e42e95a79"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.475461 11888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:32.481551 11888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:32.482398 11888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:32.482690 11888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:32.483816 11888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:32.485956 11888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:32.486070 11888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.486156 11888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:32.486214 11888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.524122 11888 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:42841
I20260807 18:22:32.524238 12059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:42841 every 8 connection(s)
I20260807 18:22:32.525781 11888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:32.531986 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 11888
I20260807 18:22:32.532213 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:22:32.532801 30323 external_mini_cluster.cc:1428] Setting key 2de6fd9122d941444551d110582d6389
I20260807 18:22:32.535341 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:32.528821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:51275 (local address 127.29.156.254:34595)
0807 18:22:32.529052 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:32.529058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:32.530621 (+ 1563us) server_negotiation.cc:407] Connection header received
0807 18:22:32.532939 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:32.532947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:32.533025 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:32.533125 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:32.534589 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:32.534594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:32.534598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:32.534621 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:32.534662 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:32.534679 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:32.534827 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:32.535060 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 18:22:32.535127 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:22:32.536232 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--builtin_ntp_servers=127.29.156.212:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:22:32.536751 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:32.529434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:34595 (local address 127.29.156.194:51275)
0807 18:22:32.530443 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:32.530491 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:32.532544 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:32.533433 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:32.533447 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:32.533555 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:32.534290 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:32.534292 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:32.534332 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:32.534420 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:32.534809 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:32.534820 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:32.534829 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:22:32.535135 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 18:22:32.535411 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":312,"threads_started":1}
I20260807 18:22:32.538196 12060 heartbeater.cc:347] Connected to a master server at 127.29.156.254:34595
I20260807 18:22:32.538523 12060 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:32.539151 12060 heartbeater.cc:511] Master 127.29.156.254:34595 requested a full tablet report, sending...
I20260807 18:22:32.540637 11627 ts_manager.cc:194] Registered new tserver with Master: eb8b2d71c99141a8b035fdad8e81d5ae (127.29.156.194:42841)
W20260807 18:22:32.897694 12064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:32.898046 12064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:32.898219 12064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:32.906018 12064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:32.906184 12064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:32.906365 12064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:32.915786 12064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:32.917243 12064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:32.919152 12064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:32.933490 12071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:32.933547 12072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:32.933609 12074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:32.934691 12064 server_base.cc:1034] running on GCE node
I20260807 18:22:32.935359 12064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:32.936429 12064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:32.937696 12064 hybrid_clock.cc:648] HybridClock initialized: now 1786126952937625 us; error 57 us; skew 500 ppm
I20260807 18:22:32.938045 12064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:22:32.940618 12064 webserver.cc:460] Webserver started at http://127.29.156.195:34465/ using document root <none> and password file <none>
I20260807 18:22:32.941393 12064 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:32.941545 12064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:32.941891 12064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:32.945070 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "71367eacb4eb4861a04601a6fa19ae95"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "83cfa225a84f0f981796785856e2f15f"
server_key_iv: "745f55cb4e37217a2fca258f9bacf6a1"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.945928 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "71367eacb4eb4861a04601a6fa19ae95"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "83cfa225a84f0f981796785856e2f15f"
server_key_iv: "745f55cb4e37217a2fca258f9bacf6a1"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.951849 12064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:22:32.955843 12080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.957569 12064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:22:32.957780 12064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "71367eacb4eb4861a04601a6fa19ae95"
format_stamp: "Formatted at 2026-08-07 18:22:32 on dist-test-slave-sb2z"
server_key: "83cfa225a84f0f981796785856e2f15f"
server_key_iv: "745f55cb4e37217a2fca258f9bacf6a1"
server_key_version: "encryptionkey@0"
I20260807 18:22:32.958081 12064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:32.964404 12064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:32.965296 12064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:32.965596 12064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:32.966610 12064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:32.968267 12064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:32.968395 12064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:32.968536 12064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:32.968629 12064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:33.006698 12064 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:35703
I20260807 18:22:33.006760 12226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:35703 every 8 connection(s)
I20260807 18:22:33.008214 12064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:33.009588 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 12064
I20260807 18:22:33.009792 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:22:33.010377 30323 external_mini_cluster.cc:1428] Setting key a9e5880f826525b23dbc52727cc8db75
I20260807 18:22:33.017899 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:33.011474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:40155 (local address 127.29.156.254:34595)
0807 18:22:33.011715 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:33.011721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:33.013492 (+ 1771us) server_negotiation.cc:407] Connection header received
0807 18:22:33.015427 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:33.015432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:33.015505 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:33.015634 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:33.017042 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:33.017049 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:33.017053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:33.017073 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:33.017115 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:33.017127 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:33.017251 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:33.017484 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 18:22:33.017560 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 18:22:33.019007 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:33.012207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:34595 (local address 127.29.156.195:40155)
0807 18:22:33.013328 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:33.013371 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:33.015153 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:33.015875 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:33.015883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:33.015976 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:33.016708 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:22:33.016711 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:33.016749 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:22:33.016839 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:33.017227 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:33.017239 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:33.017246 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:22:33.017563 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 18:22:33.017830 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":416,"threads_started":1}
I20260807 18:22:33.020797 12227 heartbeater.cc:347] Connected to a master server at 127.29.156.254:34595
I20260807 18:22:33.021226 12227 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:33.021894 12227 heartbeater.cc:511] Master 127.29.156.254:34595 requested a full tablet report, sending...
I20260807 18:22:33.023481 11627 ts_manager.cc:194] Registered new tserver with Master: 71367eacb4eb4861a04601a6fa19ae95 (127.29.156.195:35703)
I20260807 18:22:33.026690 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:22:33.033128 11886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:22:33.031317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.29.156.254:34595)
0807 18:22:33.031568 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:33.031576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:33.032114 (+ 538us) server_negotiation.cc:407] Connection header received
0807 18:22:33.032232 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:33.032236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:33.032314 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:33.032405 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:33.032857 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35540: BlockingRecv error: recv got EOF from 127.0.0.1:35540 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:22:33.034613 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11711
I20260807 18:22:33.038697 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:33.035691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49410 (local address 127.29.156.193:46259)
0807 18:22:33.036305 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:33.036312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:33.036341 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:22:33.036416 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:33.036422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:33.036640 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:33.036779 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:33.037820 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:33.037827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:33.037838 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:33.037858 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:33.037898 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:33.037922 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:33.038035 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:33.038211 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:22:33.038332 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":323,"threads_started":1}
I20260807 18:22:33.040096 11878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:33.123030 11884 heartbeater.cc:503] Master 127.29.156.254:34595 was elected leader, sending a full tablet report...
I20260807 18:22:33.124557 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11711
W20260807 18:22:33.147794 11604 connection.cc:535] server connection from 127.29.156.193:38271 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:22:33.148026 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11888
I20260807 18:22:33.151988 12244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:33.149081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.29.156.194:42841)
0807 18:22:33.149675 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:33.149683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:33.149713 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:22:33.149790 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:33.149796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:33.150099 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:33.150243 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:33.151201 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:33.151208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:33.151220 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:33.151241 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:33.151289 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:33.151304 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:33.151442 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:33.151497 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:22:33.151669 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":251,"threads_started":1}
I20260807 18:22:33.153270 12055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:33.242520 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11888
I20260807 18:22:33.266774 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 12064
I20260807 18:22:33.271359 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:33.268064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60620 (local address 127.29.156.195:35703)
0807 18:22:33.268809 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:33.268817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:33.268849 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:22:33.268915 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:33.268921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:33.269184 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:33.269371 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:33.270585 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:33.270591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:33.270602 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:22:33.270619 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:33.270660 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:22:33.270674 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:33.270866 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:33.270947 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:22:33.271051 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":376,"threads_started":1}
I20260807 18:22:33.272781 12218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:33.360460 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 12064
I20260807 18:22:33.383214 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 11582
I20260807 18:22:33.384512 11670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:33.490664 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 11582
2026-08-07T18:22:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2594 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:22:33 dist-test-slave-sb2z krb5kdc[12254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:22:33 dist-test-slave-sb2z krb5kdc[12254](info): set up 2 sockets
Aug 07 18:22:33 dist-test-slave-sb2z krb5kdc[12254](info): commencing operation
krb5kdc: starting...
W20260807 18:22:34.827818 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:22:35 dist-test-slave-sb2z krb5kdc[12254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:22:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:22:35.152097 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43677
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:35.504976 12272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:35.505287 12272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:35.505460 12272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:35.512925 12272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:22:35.513047 12272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:35.513142 12272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:35.513257 12272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:22:35.513342 12272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:22:35.523645 12272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:35.525111 12272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:35.526535 12272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:35.538687 12277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:35.538774 12278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:35.538708 12281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:35.540254 12272 server_base.cc:1034] running on GCE node
I20260807 18:22:35.541159 12272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:35.542698 12272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:35.543906 12272 hybrid_clock.cc:648] HybridClock initialized: now 1786126955543856 us; error 35 us; skew 500 ppm
I20260807 18:22:35.544355 12272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:35 dist-test-slave-sb2z krb5kdc[12254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:35.547742 12272 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:22:35.549414 12272 webserver.cc:460] Webserver started at http://127.29.156.254:42147/ using document root <none> and password file <none>
I20260807 18:22:35.550122 12272 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:35.550261 12272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:35.550606 12272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:35.553803 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "dbe0fda38d904d29b918dce57c0a0ba3"
format_stamp: "Formatted at 2026-08-07 18:22:35 on dist-test-slave-sb2z"
server_key: "296589b2232e20126dc70b0487a3a2e2"
server_key_iv: "fac9cc85e4992e1a422c9c58ab8ae094"
server_key_version: "encryptionkey@0"
I20260807 18:22:35.554608 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "dbe0fda38d904d29b918dce57c0a0ba3"
format_stamp: "Formatted at 2026-08-07 18:22:35 on dist-test-slave-sb2z"
server_key: "296589b2232e20126dc70b0487a3a2e2"
server_key_iv: "fac9cc85e4992e1a422c9c58ab8ae094"
server_key_version: "encryptionkey@0"
I20260807 18:22:35.559942 12272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:22:35.563836 12293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:35.565214 12272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:22:35.565418 12272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "dbe0fda38d904d29b918dce57c0a0ba3"
format_stamp: "Formatted at 2026-08-07 18:22:35 on dist-test-slave-sb2z"
server_key: "296589b2232e20126dc70b0487a3a2e2"
server_key_iv: "fac9cc85e4992e1a422c9c58ab8ae094"
server_key_version: "encryptionkey@0"
I20260807 18:22:35.565637 12272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:35.600768 12272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:35.605211 12272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:35.605531 12272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:35.625368 12272 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43677
I20260807 18:22:35.625412 12360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43677 every 8 connection(s)
I20260807 18:22:35.626792 12272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:22:35.630579 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:35.634691 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 12272
I20260807 18:22:35.634977 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:22:35.635835 30323 external_mini_cluster.cc:1428] Setting key 034fa39809040a3847ed212ead8988c8
I20260807 18:22:35.643394 12361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: Bootstrap starting.
I20260807 18:22:35.647245 12361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:35.648443 12361 log.cc:824] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:35.651690 12361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: No bootstrap required, opened a new log
Aug 07 18:22:35 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:35.663652 12361 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } }
I20260807 18:22:35.664238 12361 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:35.664359 12361 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe0fda38d904d29b918dce57c0a0ba3, State: Initialized, Role: FOLLOWER
I20260807 18:22:35.664824 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:35.638195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.29.156.254:43677)
0807 18:22:35.639054 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:35.639074 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:35.639136 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:22:35.641745 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:35.641775 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:35.642274 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:35.642723 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:35.644557 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:35.646652 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:35.648283 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:35.649390 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:35.649544 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:35.654508 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:35.654526 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:35.654546 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:35.654616 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:35.659223 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:35.660365 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:35.660375 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:35.660385 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:35.660640 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:35.661094 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:35.661102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:35.661105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:35.661941 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:35.662290 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:35.662776 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:22:35.663243 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":370,"threads_started":1}
I20260807 18:22:35.665062 12361 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } }
I20260807 18:22:35.665355 12361 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:22:35.665516 12361 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:22:35.665704 12361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:35.668526 12361 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } }
I20260807 18:22:35.668956 12361 leader_election.cc:302] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbe0fda38d904d29b918dce57c0a0ba3; no voters:
I20260807 18:22:35.669606 12361 leader_election.cc:288] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:22:35.669804 12367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:35.670686 12367 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [term 1 LEADER]: Becoming Leader. State: Replica: dbe0fda38d904d29b918dce57c0a0ba3, State: Running, Role: LEADER
I20260807 18:22:35.671247 12367 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } }
I20260807 18:22:35.671907 12361 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:22:35.674454 12369 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } } }
I20260807 18:22:35.675035 12369 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:35.674484 12370 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe0fda38d904d29b918dce57c0a0ba3. Latest consensus state: current_term: 1 leader_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe0fda38d904d29b918dce57c0a0ba3" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43677 } } }
I20260807 18:22:35.675491 12370 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:22:35.677093 12373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:22:35.682960 12373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:22:35.693928 12373 catalog_manager.cc:1347] Generated new cluster ID: f9b7f26a9756453b89f4fec476c7e4a3
I20260807 18:22:35.694069 12373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:22:35.712980 12373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:22:35.714383 12373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:22:35.724156 12373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: Generated new TSK 0
I20260807 18:22:35.725080 12373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:22:35.823678 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:36.182405 12399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:36.182710 12399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:36.182871 12399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:36.190431 12399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:36.190550 12399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:36.190739 12399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:22:36.201094 12399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:36.202503 12399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:36.204007 12399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:36.217976 12404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:36.217888 12407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:36.218000 12405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:36.218930 12399 server_base.cc:1034] running on GCE node
I20260807 18:22:36.219698 12399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:36.220738 12399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:36.221987 12399 hybrid_clock.cc:648] HybridClock initialized: now 1786126956221932 us; error 80 us; skew 500 ppm
I20260807 18:22:36.222366 12399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:36 dist-test-slave-sb2z krb5kdc[12254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:36.225811 12399 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:22:36.227509 12399 webserver.cc:460] Webserver started at http://127.29.156.193:40851/ using document root <none> and password file <none>
I20260807 18:22:36.228300 12399 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:36.228408 12399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:36.228763 12399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:36.231850 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "498b97b817764efaadd7cfe31daa61c9"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "80329ea4c35a3905981794316e562880"
server_key_iv: "5abe9b9878606ef83df5aa71cfd4696e"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.232709 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "498b97b817764efaadd7cfe31daa61c9"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "80329ea4c35a3905981794316e562880"
server_key_iv: "5abe9b9878606ef83df5aa71cfd4696e"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.238257 12399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:22:36.242024 12417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.243494 12399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:22:36.243732 12399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "498b97b817764efaadd7cfe31daa61c9"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "80329ea4c35a3905981794316e562880"
server_key_iv: "5abe9b9878606ef83df5aa71cfd4696e"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.243942 12399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:36.265089 12399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:36.269706 12399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:36.270092 12399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:36.271133 12399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:36.272778 12399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:36.272867 12399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.272955 12399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:36.273099 12399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.309604 12399 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:36871
I20260807 18:22:36.309733 12572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:36871 every 8 connection(s)
I20260807 18:22:36.311105 12399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:22:36.316462 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 12399
I20260807 18:22:36.316715 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:22:36.317618 30323 external_mini_cluster.cc:1428] Setting key aa18b48ee970132fb23dbe1b447c02aa
Aug 07 18:22:36 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:36.333319 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:36.314207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:34619 (local address 127.29.156.254:43677)
0807 18:22:36.314706 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:36.314710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:36.315720 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 18:22:36.318605 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:36.318610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:36.318681 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:36.318814 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:36.320618 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.321359 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.322924 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.323345 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.323485 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:36.327418 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:36.327424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:36.327427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:36.327467 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:36.330215 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:36.330962 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:36.330973 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:36.330975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:36.331097 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:36.331565 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:36.331570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:36.331574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:36.331843 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:36.332004 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:36.332721 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 18:22:36.332954 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":276,"threads_started":1}
I20260807 18:22:36.334444 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:36.314741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43677 (local address 127.29.156.193:34619)
0807 18:22:36.315543 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:36.315591 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:36.318300 (+ 2709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:36.319115 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:36.319124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:36.319855 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:36.320437 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:36.320454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.321535 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.321547 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:36.322767 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:36.322778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.323625 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.323634 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:36.323866 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:36.324580 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:36.324608 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:36.327168 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:36.330374 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:36.330383 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:36.330399 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:36.330829 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:36.331210 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:36.331215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:36.331219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:36.331414 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:36.331986 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:36.331993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:36.332332 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 18:22:36.332603 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 18:22:36.333150 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":311,"threads_started":1}
I20260807 18:22:36.336185 12573 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43677
I20260807 18:22:36.336508 12573 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:36.337270 12573 heartbeater.cc:511] Master 127.29.156.254:43677 requested a full tablet report, sending...
I20260807 18:22:36.339752 12316 ts_manager.cc:194] Registered new tserver with Master: 498b97b817764efaadd7cfe31daa61c9 (127.29.156.193:36871)
I20260807 18:22:36.342311 12316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:34619
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:22:36.392916 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:36.742013 12582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:36.742334 12582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:36.742434 12582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:36.752408 12582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:36.752556 12582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:36.752678 12582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:22:36.762987 12582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:36.764529 12582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:36.765887 12582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:22:36.780696 12594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:36.780812 12590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:36.780786 12592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:36.781796 12582 server_base.cc:1034] running on GCE node
I20260807 18:22:36.782583 12582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:36.783730 12582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:36.784983 12582 hybrid_clock.cc:648] HybridClock initialized: now 1786126956784933 us; error 42 us; skew 500 ppm
I20260807 18:22:36.785359 12582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:36 dist-test-slave-sb2z krb5kdc[12254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:36.788836 12582 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:22:36.790488 12582 webserver.cc:460] Webserver started at http://127.29.156.194:42867/ using document root <none> and password file <none>
I20260807 18:22:36.791225 12582 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:36.791385 12582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:36.791781 12582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:36.794909 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "2acb7226f6c04259a06ac93f170b70d5"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "d9a6b7374ff7068aea0f7d3300dc7152"
server_key_iv: "2c8e19557b6420fa884e45c862c180ae"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.795848 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "2acb7226f6c04259a06ac93f170b70d5"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "d9a6b7374ff7068aea0f7d3300dc7152"
server_key_iv: "2c8e19557b6420fa884e45c862c180ae"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.801129 12582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:22:36.804720 12603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.806144 12582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:22:36.806356 12582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "2acb7226f6c04259a06ac93f170b70d5"
format_stamp: "Formatted at 2026-08-07 18:22:36 on dist-test-slave-sb2z"
server_key: "d9a6b7374ff7068aea0f7d3300dc7152"
server_key_iv: "2c8e19557b6420fa884e45c862c180ae"
server_key_version: "encryptionkey@0"
I20260807 18:22:36.806587 12582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:36.831751 12582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:36.836462 12582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:36.836784 12582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:36.837790 12582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:36.839169 12582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:36.839273 12582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.839424 12582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:36.839500 12582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:36.874413 12582 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:46297
I20260807 18:22:36.874446 12761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:46297 every 8 connection(s)
I20260807 18:22:36.875823 12582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:22:36.886049 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 12582
I20260807 18:22:36.886267 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:22:36.886734 30323 external_mini_cluster.cc:1428] Setting key f38c9d1d65dd2ca0c02557192af65b78
Aug 07 18:22:36 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:36.898187 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:36.878486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:39083 (local address 127.29.156.254:43677)
0807 18:22:36.878981 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:36.878985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:36.879941 (+ 956us) server_negotiation.cc:407] Connection header received
0807 18:22:36.881833 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:36.881840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:36.881933 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:36.882086 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:36.883587 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.884313 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.885852 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.886396 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.886541 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:36.890672 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:36.890683 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:36.890691 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:36.890739 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:36.894632 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:36.895463 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:36.895471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:36.895477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:36.895669 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:36.896096 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:36.896109 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:36.896115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:36.896515 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:36.896666 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:36.897431 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 18:22:36.897693 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":271,"threads_started":1}
I20260807 18:22:36.898898 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:36.878967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43677 (local address 127.29.156.194:39083)
0807 18:22:36.879777 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:36.879815 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:36.881536 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:36.882319 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:36.882326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:36.882833 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:36.883384 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:36.883401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.884480 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.884485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:36.885704 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:36.885716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:36.886645 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:36.886654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:36.886926 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:36.887690 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:36.887712 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:36.890420 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:36.894812 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:36.894820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:36.894834 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:36.895333 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:36.895787 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:36.895791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:36.895793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:36.895985 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:36.896666 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:36.896673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:36.897013 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 18:22:36.897298 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 18:22:36.897702 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":287,"threads_started":1}
I20260807 18:22:36.900497 12762 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43677
I20260807 18:22:36.900847 12762 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:36.901783 12762 heartbeater.cc:511] Master 127.29.156.254:43677 requested a full tablet report, sending...
I20260807 18:22:36.903414 12316 ts_manager.cc:194] Registered new tserver with Master: 2acb7226f6c04259a06ac93f170b70d5 (127.29.156.194:46297)
I20260807 18:22:36.904305 12316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:39083
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:22:36.959477 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:22:37.308825 12770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:22:37.309154 12770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:22:37.309338 12770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:22:37.317271 12770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:22:37.317420 12770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:22:37.317679 12770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:22:37.329167 12770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:22:37.330595 12770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:22:37.332106 12770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:22:37.345715 12573 heartbeater.cc:503] Master 127.29.156.254:43677 was elected leader, sending a full tablet report...
W20260807 18:22:37.346045 12780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:37.346742 12779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:22:37.346833 12782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:22:37.347491 12770 server_base.cc:1034] running on GCE node
I20260807 18:22:37.348305 12770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:22:37.349267 12770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:22:37.350500 12770 hybrid_clock.cc:648] HybridClock initialized: now 1786126957350455 us; error 66 us; skew 500 ppm
I20260807 18:22:37.350831 12770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:22:37 dist-test-slave-sb2z krb5kdc[12254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126957, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:22:37.354290 12770 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:22:37.355845 12770 webserver.cc:460] Webserver started at http://127.29.156.195:36161/ using document root <none> and password file <none>
I20260807 18:22:37.356551 12770 fs_manager.cc:359] Metadata directory not provided
I20260807 18:22:37.356707 12770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:22:37.357028 12770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:22:37.360066 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "62fe64ad4b194710afba717afd71e87a"
format_stamp: "Formatted at 2026-08-07 18:22:37 on dist-test-slave-sb2z"
server_key: "36fea686d0cc37594d495e082c073c31"
server_key_iv: "ef1b0a2e55debff4626a7ea3c3afcf06"
server_key_version: "encryptionkey@0"
I20260807 18:22:37.360864 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "62fe64ad4b194710afba717afd71e87a"
format_stamp: "Formatted at 2026-08-07 18:22:37 on dist-test-slave-sb2z"
server_key: "36fea686d0cc37594d495e082c073c31"
server_key_iv: "ef1b0a2e55debff4626a7ea3c3afcf06"
server_key_version: "encryptionkey@0"
I20260807 18:22:37.366199 12770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:22:37.369827 12790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:37.371212 12770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:22:37.371392 12770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "62fe64ad4b194710afba717afd71e87a"
format_stamp: "Formatted at 2026-08-07 18:22:37 on dist-test-slave-sb2z"
server_key: "36fea686d0cc37594d495e082c073c31"
server_key_iv: "ef1b0a2e55debff4626a7ea3c3afcf06"
server_key_version: "encryptionkey@0"
I20260807 18:22:37.371665 12770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:22:37.403829 12770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:22:37.408257 12770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:22:37.408596 12770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:22:37.409595 12770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:22:37.411108 12770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:22:37.411216 12770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:37.411373 12770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:22:37.411458 12770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:22:37.446336 12770 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45143
I20260807 18:22:37.446391 12941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45143 every 8 connection(s)
I20260807 18:22:37.447842 12770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:22:37.453433 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 12770
I20260807 18:22:37.453655 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:22:37.454221 30323 external_mini_cluster.cc:1428] Setting key 1cd48cacfae61d7367637422062d161b
Aug 07 18:22:37 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126957, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:22:37.470260 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.450883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43857 (local address 127.29.156.254:43677)
0807 18:22:37.451389 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.451393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.452504 (+ 1111us) server_negotiation.cc:407] Connection header received
0807 18:22:37.455098 (+ 2594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.455105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.455196 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.455327 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:37.457008 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.457804 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.459382 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.459837 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.459976 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.463615 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:37.463621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:37.463625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:37.463661 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:37.466663 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.467506 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.467514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.467518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.467704 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.468219 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.468225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.468229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.468561 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:37.468727 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.469567 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.469829 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":246,"threads_started":1}
I20260807 18:22:37.471372 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.451471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43677 (local address 127.29.156.195:43857)
0807 18:22:37.452328 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.452368 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.454790 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.455664 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.455673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.456243 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:22:37.456827 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.456845 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.457981 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.457985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.459232 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.459245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.460068 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.460077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.460332 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.460975 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:22:37.460998 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:22:37.463293 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:22:37.466835 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:37.466845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:37.466873 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:37.467356 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:37.467836 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:22:37.467842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:22:37.467847 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:22:37.468086 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:22:37.468719 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:22:37.468727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:22:37.469120 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.469451 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.470110 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":371,"threads_started":1}
I20260807 18:22:37.472888 12942 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43677
I20260807 18:22:37.473228 12942 heartbeater.cc:464] Registering TS with master...
I20260807 18:22:37.473976 12942 heartbeater.cc:511] Master 127.29.156.254:43677 requested a full tablet report, sending...
I20260807 18:22:37.475569 12316 ts_manager.cc:194] Registered new tserver with Master: 62fe64ad4b194710afba717afd71e87a (127.29.156.195:45143)
I20260807 18:22:37.476457 12316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:43857
I20260807 18:22:37.482163 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:22:37.500308 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.486798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.29.156.254:43677)
0807 18:22:37.487086 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.487095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.487427 (+ 332us) server_negotiation.cc:407] Connection header received
0807 18:22:37.487684 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.487689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.487766 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.487872 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:37.489340 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.490101 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.491372 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.491865 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.492040 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.494714 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:37.494720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:37.494723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:37.494759 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:37.497508 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.498210 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.498217 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.498222 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.498354 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.498740 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.498746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.498749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.499090 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:37.499352 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.499748 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.499982 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:22:37.504971 12316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:22:37.512743 12316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:22:37.550676 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.525792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46297 (local address 127.0.0.1:35168)
0807 18:22:37.538525 (+ 12733us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.538574 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.538821 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.541521 (+ 2700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.541530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.541577 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:37.542249 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.542269 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.544523 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.544551 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.547465 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.547482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.549414 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.549421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.549638 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.549703 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.549863 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.550076 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12379,"mutex_wait_us":204,"spinlock_wait_cycles":16512,"thread_start_us":12408,"threads_started":1}
I20260807 18:22:37.551694 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.525868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45143 (local address 127.0.0.1:40150)
0807 18:22:37.538525 (+ 12657us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.538574 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.538819 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.541521 (+ 2702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.541530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.541575 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:37.543505 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.543546 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.545649 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.545656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.548231 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.548248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.550635 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.550644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.550924 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.550962 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.551065 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.551219 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12517,"mutex_wait_us":74,"spinlock_wait_cycles":15360,"thread_start_us":12091,"threads_started":1}
I20260807 18:22:37.551779 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.525919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35168 (local address 127.29.156.194:46297)
0807 18:22:37.540478 (+ 14559us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.540489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.540524 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:22:37.540648 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.540656 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.540955 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.541192 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:37.542511 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.544311 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.547685 (+ 3374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.549189 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.549400 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.549615 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.549918 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.551328 (+ 1410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14371,"thread_start_us":15654,"threads_started":1}
I20260807 18:22:37.552281 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.526335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.29.156.195:45143)
0807 18:22:37.540515 (+ 14180us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.540523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.540556 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:22:37.540650 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.540657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.540957 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.541192 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:37.543767 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.545515 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.548418 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.550069 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.550273 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.550450 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.551700 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.551892 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13997,"thread_start_us":13814,"threads_started":1}
I20260807 18:22:37.553821 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.540641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52610)
0807 18:22:37.544392 (+ 3751us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.544458 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.544636 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.545351 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.545357 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.545391 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:37.545958 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.545973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.548192 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.548202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.550589 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.550617 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.553130 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.553137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.553325 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.553362 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.553461 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.553562 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3576,"thread_start_us":3432,"threads_started":1}
I20260807 18:22:37.554641 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.539516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52610 (local address 127.29.156.193:36871)
0807 18:22:37.544053 (+ 4537us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.544065 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.544617 (+ 552us) server_negotiation.cc:407] Connection header received
0807 18:22:37.544831 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.544840 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.545113 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.545310 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:37.546239 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.548004 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.550966 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.552878 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.553111 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.554175 (+ 1064us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.554244 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.554364 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4333,"thread_start_us":4141,"threads_started":1}
I20260807 18:22:37.556514 12852 tablet_service.cc:1467] Processing CreateTablet for tablet c38977443459431197237fa564823cb7 (DEFAULT_TABLE table=test-table [id=7c69274db166405887b5e31e36304979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:37.556524 12677 tablet_service.cc:1467] Processing CreateTablet for tablet c38977443459431197237fa564823cb7 (DEFAULT_TABLE table=test-table [id=7c69274db166405887b5e31e36304979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:37.558176 12852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38977443459431197237fa564823cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:37.558176 12677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38977443459431197237fa564823cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:37.558183 12482 tablet_service.cc:1467] Processing CreateTablet for tablet c38977443459431197237fa564823cb7 (DEFAULT_TABLE table=test-table [id=7c69274db166405887b5e31e36304979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:22:37.559430 12482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38977443459431197237fa564823cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:22:37.572494 12970 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Bootstrap starting.
I20260807 18:22:37.574822 12972 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Bootstrap starting.
I20260807 18:22:37.576180 12971 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Bootstrap starting.
I20260807 18:22:37.577518 12970 tablet_bootstrap.cc:654] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:37.579142 12970 log.cc:824] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:37.579959 12972 tablet_bootstrap.cc:654] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:37.580248 12971 tablet_bootstrap.cc:654] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Neither blocks nor log segments found. Creating new log.
I20260807 18:22:37.581358 12972 log.cc:824] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:37.581667 12971 log.cc:824] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:22:37.583040 12970 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: No bootstrap required, opened a new log
I20260807 18:22:37.583351 12970 ts_tablet_manager.cc:1397] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:22:37.584995 12972 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: No bootstrap required, opened a new log
I20260807 18:22:37.585000 12971 tablet_bootstrap.cc:492] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: No bootstrap required, opened a new log
I20260807 18:22:37.585274 12972 ts_tablet_manager.cc:1397] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 18:22:37.585297 12971 ts_tablet_manager.cc:1397] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:22:37.593822 12971 raft_consensus.cc:348] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.594275 12971 raft_consensus.cc:374] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:37.594398 12971 raft_consensus.cc:729] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acb7226f6c04259a06ac93f170b70d5, State: Initialized, Role: FOLLOWER
I20260807 18:22:37.595126 12971 consensus_queue.cc:260] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.595743 12970 raft_consensus.cc:348] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.596226 12970 raft_consensus.cc:374] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:37.596341 12970 raft_consensus.cc:729] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498b97b817764efaadd7cfe31daa61c9, State: Initialized, Role: FOLLOWER
I20260807 18:22:37.596604 12762 heartbeater.cc:503] Master 127.29.156.254:43677 was elected leader, sending a full tablet report...
I20260807 18:22:37.597086 12970 consensus_queue.cc:260] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.597344 12971 ts_tablet_manager.cc:1428] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260807 18:22:37.597373 12972 raft_consensus.cc:348] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.597924 12972 raft_consensus.cc:374] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:22:37.598088 12972 raft_consensus.cc:729] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fe64ad4b194710afba717afd71e87a, State: Initialized, Role: FOLLOWER
I20260807 18:22:37.599365 12970 ts_tablet_manager.cc:1428] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260807 18:22:37.599936 12972 consensus_queue.cc:260] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.601366 12942 heartbeater.cc:503] Master 127.29.156.254:43677 was elected leader, sending a full tablet report...
I20260807 18:22:37.602172 12972 ts_tablet_manager.cc:1428] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260807 18:22:37.617981 12980 raft_consensus.cc:482] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:22:37.618345 12980 raft_consensus.cc:504] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.620381 12980 leader_election.cc:288] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2acb7226f6c04259a06ac93f170b70d5 (127.29.156.194:46297), 498b97b817764efaadd7cfe31daa61c9 (127.29.156.193:36871)
I20260807 18:22:37.626519 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.620861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:56975 (local address 127.29.156.194:46297)
0807 18:22:37.621097 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.621115 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.621468 (+ 353us) server_negotiation.cc:407] Connection header received
0807 18:22:37.621658 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.621666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.621758 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.621870 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:37.622243 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.622974 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.624666 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.625496 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.625664 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.625737 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.626131 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.626272 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:22:37.626533 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.620974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46297 (local address 127.29.156.195:56975)
0807 18:22:37.621292 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.621318 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.621481 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.621897 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.621900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.621922 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:37.622084 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.622091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.623138 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.623143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.624480 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.624493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.625659 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.625665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.625946 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.625986 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.626089 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.626236 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":43}
I20260807 18:22:37.627686 12701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38977443459431197237fa564823cb7" candidate_uuid: "62fe64ad4b194710afba717afd71e87a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2acb7226f6c04259a06ac93f170b70d5" is_pre_election: true
W20260807 18:22:37.628026 12763 tablet.cc:2369] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:22:37.628288 12701 raft_consensus.cc:2454] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fe64ad4b194710afba717afd71e87a in term 0.
I20260807 18:22:37.628432 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.620872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:60307 (local address 127.29.156.193:36871)
0807 18:22:37.621134 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.621142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.622165 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 18:22:37.622356 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.622362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.622432 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.622524 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:22:37.623072 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.623857 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.625998 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.627225 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.627397 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.627459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.628044 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.628190 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:22:37.629191 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.621117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.29.156.195:60307)
0807 18:22:37.622062 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.622077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:22:37.622217 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:22:37.622540 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:22:37.622545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:22:37.622567 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:22:37.622909 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.622917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.624000 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.624005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.625831 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:22:37.625846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.627635 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.627640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:22:37.627833 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.627870 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:22:37.628623 (+ 753us) client_negotiation.cc:240] Negotiation successful
0807 18:22:37.628760 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":26,"thread_start_us":476,"threads_started":1}
I20260807 18:22:37.629513 12796 leader_election.cc:302] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2acb7226f6c04259a06ac93f170b70d5, 62fe64ad4b194710afba717afd71e87a; no voters:
I20260807 18:22:37.629956 12980 raft_consensus.cc:2781] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:22:37.630111 12980 raft_consensus.cc:482] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:22:37.630203 12980 raft_consensus.cc:3037] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:37.630208 12509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38977443459431197237fa564823cb7" candidate_uuid: "62fe64ad4b194710afba717afd71e87a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498b97b817764efaadd7cfe31daa61c9" is_pre_election: true
I20260807 18:22:37.630748 12509 raft_consensus.cc:2454] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fe64ad4b194710afba717afd71e87a in term 0.
I20260807 18:22:37.633489 12980 raft_consensus.cc:504] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.634255 12980 leader_election.cc:288] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [CANDIDATE]: Term 1 election: Requested vote from peers 2acb7226f6c04259a06ac93f170b70d5 (127.29.156.194:46297), 498b97b817764efaadd7cfe31daa61c9 (127.29.156.193:36871)
I20260807 18:22:37.634764 12701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38977443459431197237fa564823cb7" candidate_uuid: "62fe64ad4b194710afba717afd71e87a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2acb7226f6c04259a06ac93f170b70d5"
I20260807 18:22:37.634764 12509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38977443459431197237fa564823cb7" candidate_uuid: "62fe64ad4b194710afba717afd71e87a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498b97b817764efaadd7cfe31daa61c9"
I20260807 18:22:37.634976 12509 raft_consensus.cc:3037] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:37.635005 12701 raft_consensus.cc:3037] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:22:37.637388 12701 raft_consensus.cc:2454] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fe64ad4b194710afba717afd71e87a in term 1.
I20260807 18:22:37.637876 12796 leader_election.cc:302] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2acb7226f6c04259a06ac93f170b70d5, 62fe64ad4b194710afba717afd71e87a; no voters:
I20260807 18:22:37.638226 12980 raft_consensus.cc:2781] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:22:37.638223 12509 raft_consensus.cc:2454] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fe64ad4b194710afba717afd71e87a in term 1.
I20260807 18:22:37.639315 12980 raft_consensus.cc:686] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 1 LEADER]: Becoming Leader. State: Replica: 62fe64ad4b194710afba717afd71e87a, State: Running, Role: LEADER
I20260807 18:22:37.639876 12980 consensus_queue.cc:237] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } }
I20260807 18:22:37.644461 12315 catalog_manager.cc:5495] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a reported cstate change: term changed from 0 to 1, leader changed from <none> to 62fe64ad4b194710afba717afd71e87a (127.29.156.195). New cstate: current_term: 1 leader_uuid: "62fe64ad4b194710afba717afd71e87a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62fe64ad4b194710afba717afd71e87a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 } health_report { overall_health: UNKNOWN } } }
I20260807 18:22:37.677439 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.671287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.29.156.195:45143)
0807 18:22:37.671557 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.671568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.671711 (+ 143us) server_negotiation.cc:407] Connection header received
0807 18:22:37.671940 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.671950 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.672074 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.672217 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:22:37.672581 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.673448 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.674869 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.675371 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.675476 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.676237 (+ 761us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:22:37.676781 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:22:37.676964 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.677060 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.677130 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:22:37.686146 12509 raft_consensus.cc:1261] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 1 FOLLOWER]: Refusing update from remote peer 62fe64ad4b194710afba717afd71e87a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:37.686216 12701 raft_consensus.cc:1261] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 1 FOLLOWER]: Refusing update from remote peer 62fe64ad4b194710afba717afd71e87a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:22:37.687187 12982 consensus_queue.cc:1035] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [LEADER]: Connected to new peer: Peer: permanent_uuid: "498b97b817764efaadd7cfe31daa61c9" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:37.687593 12980 consensus_queue.cc:1035] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2acb7226f6c04259a06ac93f170b70d5" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:22:37.698974 12987 mvcc.cc:205] Tried to move back new op lower bound from 7315976018672410624 to 7315976018497036288. Current Snapshot: MvccSnapshot[applied={T|T < 7315976018672410624}]
I20260807 18:22:37.699743 12988 mvcc.cc:205] Tried to move back new op lower bound from 7315976018672410624 to 7315976018497036288. Current Snapshot: MvccSnapshot[applied={T|T < 7315976018672410624}]
I20260807 18:22:37.712949 12315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:37.713233 12315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:22:37.716790 12315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbe0fda38d904d29b918dce57c0a0ba3: Sending DeleteTablet for 3 replicas of tablet c38977443459431197237fa564823cb7
I20260807 18:22:37.718184 12677 tablet_service.cc:1514] Processing DeleteTablet for tablet c38977443459431197237fa564823cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:37 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:35168
I20260807 18:22:37.718235 12852 tablet_service.cc:1514] Processing DeleteTablet for tablet c38977443459431197237fa564823cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:37 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:40150
I20260807 18:22:37.718231 12482 tablet_service.cc:1514] Processing DeleteTablet for tablet c38977443459431197237fa564823cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:22:37 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:52610
I20260807 18:22:37.720335 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 12399
I20260807 18:22:37.731256 12998 tablet_replica.cc:330] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: stopping tablet replica
I20260807 18:22:37.732115 13000 tablet_replica.cc:330] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: stopping tablet replica
I20260807 18:22:37.732681 13000 raft_consensus.cc:2229] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:22:37.733525 13000 raft_consensus.cc:2258] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:37.734862 12998 raft_consensus.cc:2229] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:37.735576 12998 raft_consensus.cc:2258] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:22:37.737138 13000 ts_tablet_manager.cc:1905] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:37.738219 12998 ts_tablet_manager.cc:1905] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:37.740191 12999 tablet_replica.cc:330] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: stopping tablet replica
I20260807 18:22:37.740597 12999 raft_consensus.cc:2229] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:22:37.741139 12999 raft_consensus.cc:2258] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 18:22:37 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:22:37.746007 13000 ts_tablet_manager.cc:1918] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:37.746294 13000 log.cc:1196] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/c38977443459431197237fa564823cb7
I20260807 18:22:37.746570 12998 ts_tablet_manager.cc:1918] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:37.746764 12998 log.cc:1196] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/c38977443459431197237fa564823cb7
I20260807 18:22:37.747179 13000 ts_tablet_manager.cc:1939] T c38977443459431197237fa564823cb7 P 62fe64ad4b194710afba717afd71e87a: Deleting consensus metadata
I20260807 18:22:37.747398 12998 ts_tablet_manager.cc:1939] T c38977443459431197237fa564823cb7 P 2acb7226f6c04259a06ac93f170b70d5: Deleting consensus metadata
I20260807 18:22:37.749402 12298 catalog_manager.cc:4841] TS 2acb7226f6c04259a06ac93f170b70d5 (127.29.156.194:46297): tablet c38977443459431197237fa564823cb7 (table test-table [id=7c69274db166405887b5e31e36304979]) successfully deleted
I20260807 18:22:37.749771 12297 catalog_manager.cc:4841] TS 62fe64ad4b194710afba717afd71e87a (127.29.156.195:45143): tablet c38977443459431197237fa564823cb7 (table test-table [id=7c69274db166405887b5e31e36304979]) successfully deleted
I20260807 18:22:37.750619 12999 ts_tablet_manager.cc:1905] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:22:37.755107 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.721366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.29.156.193:36871)
0807 18:22:37.721629 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.721638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.732091 (+ 10453us) server_negotiation.cc:407] Connection header received
0807 18:22:37.732361 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.732376 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.732460 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.732531 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:37.734118 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.735368 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.737224 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.737949 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.738059 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.744116 (+ 6057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:37.744137 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:37.744156 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:37.744211 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:37.748626 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.751908 (+ 3282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.751918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.751922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.752190 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.752798 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.752804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.752808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.753364 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:37.753706 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.754351 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.754653 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:22:37.756896 12568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:37.872295 12999 ts_tablet_manager.cc:1918] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:22:37.872654 12999 log.cc:1196] T c38977443459431197237fa564823cb7 P 498b97b817764efaadd7cfe31daa61c9: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/c38977443459431197237fa564823cb7
I20260807 18:22:37.873139 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 12399
W20260807 18:22:37.907335 12297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.156.193:36871 (error 108)
I20260807 18:22:37.907652 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 12582
I20260807 18:22:37.910235 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.909667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52642)
0807 18:22:37.909916 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.910052 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260807 18:22:37.910655 12297 catalog_manager.cc:4584] TS 498b97b817764efaadd7cfe31daa61c9 (127.29.156.193:36871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c38977443459431197237fa564823cb7: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Aug 07 18:22:37 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:22:37.926558 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.908885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.29.156.194:46297)
0807 18:22:37.909069 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:37.909075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:37.909097 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:22:37.909161 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:37.909167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:37.909241 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:37.909349 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:37.911394 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.912785 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.914071 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:37.914695 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:37.914831 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:37.919174 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:37.919197 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:37.919219 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:37.919266 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:37.923369 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.924238 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.924248 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.924252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.924405 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:37.924773 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:37.924778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:37.924781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:37.925309 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:37.925608 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:37.925893 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:22:37.926161 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:22:37.927397 12756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:37.964474 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.963891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52648)
0807 18:22:37.964143 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.964268 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:22:37.965751 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:37.965315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52650)
0807 18:22:37.965498 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:37.965596 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:22:38.019731 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 12582
I20260807 18:22:38.026886 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.026367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52664)
0807 18:22:38.026625 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:38.026714 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:22:38.028139 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.027649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52676)
0807 18:22:38.027866 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:38.027949 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 18:22:38.047318 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 12770
Aug 07 18:22:38 dist-test-slave-sb2z krb5kdc[12254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:22:38.066860 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.048345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.29.156.195:45143)
0807 18:22:38.048576 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:22:38.048582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:22:38.048640 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:22:38.048718 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:22:38.048724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:22:38.048811 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:22:38.048939 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:22:38.050200 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:38.051434 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:38.052925 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:22:38.053606 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:22:38.053760 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:22:38.058736 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:22:38.058759 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:22:38.058782 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:22:38.058845 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:22:38.063280 (+ 4435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:38.064384 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:38.064394 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:38.064398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:38.064610 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:22:38.065045 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:22:38.065053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:22:38.065057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:22:38.065582 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:22:38.065962 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:22:38.066137 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:22:38.066389 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:22:38.067827 12936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:38.109582 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.108946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52680)
0807 18:22:38.109279 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:38.109381 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 18:22:38.110762 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.110316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52690)
0807 18:22:38.110561 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:38.110638 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 18:22:38.196564 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 12770
I20260807 18:22:38.224519 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 12272
I20260807 18:22:38.225798 12355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:22:38.340869 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:22:38.340269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:36871 (local address 127.0.0.1:52704)
0807 18:22:38.340481 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:22:38.340626 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:22:38.341280 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 12272
2026-08-07T18:22:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4848 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:22:38 dist-test-slave-sb2z krb5kdc[13010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:22:38 dist-test-slave-sb2z krb5kdc[13010](info): set up 2 sockets
Aug 07 18:22:38 dist-test-slave-sb2z krb5kdc[13010](info): commencing operation
krb5kdc: starting...
W20260807 18:22:39.704715 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:22:39 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:22:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:22:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangeradmin_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangeradmin_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangerlookup_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangerlookup_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
I20260807 18:22:40.169306 30323 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyv84zy/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/postgres -l logfile start
2026-08-07 18:22:43.105 UTC [13047] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:22:43.105 UTC [13047] LOG: listening on IPv4 address "127.29.156.212", port 48973
2026-08-07 18:22:43.109 UTC [13047] LOG: listening on Unix socket "/tmp/.s.PGSQL.48973"
2026-08-07 18:22:43.136 UTC [13049] LOG: database system was shut down at 2026-08-07 18:22:41 UTC
2026-08-07 18:22:43.140 UTC [13047] LOG: database system is ready to accept connections
I20260807 18:22:44.338220 30323 mini_postgres.cc:96] Postgres bound to 48973
2026-08-07 18:22:44.368 UTC [13059] FATAL: database "slave" does not exist
127.29.156.212:48973 - accepting connections
I20260807 18:22:44.368937 30323 mini_ranger.cc:161] Starting Ranger...
I20260807 18:22:44.407747 30323 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:22:45.003835 30323 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:22:45.004014 30323 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin
2026-08-07 18:22:45,474 [I] DB FLAVOR :POSTGRES
2026-08-07 18:22:45,474 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:22:45,474 [I] Checking connection..
2026-08-07 18:22:45,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:22:45,801 [I] Checking connection passed.
2026-08-07 18:22:45,801 [I] --------- Verifying version history table ---------
2026-08-07 18:22:45,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:22:46,090 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:22:46,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:22:46,353 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:22:46,354 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:22:46,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:22:46.610 UTC [13148] WARNING: there is no transaction in progress
2026-08-07 18:22:46,619 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:22:46,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:22:46,900 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:22:46,900 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:22:46,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:22:47,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:22:47,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-07 18:22:47,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:22:47,976 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:22:47,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:22:48,243 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:22:48,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:22:48,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:22:48,763 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:22:48,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:22:49.819 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.836 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.853 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.871 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.888 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.911 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.968 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.975 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:49.990 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.000 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.011 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.026 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.036 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.042 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.049 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50.056 UTC [13293] WARNING: there is no transaction in progress
2026-08-07 18:22:50,770 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:22:50,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:22:51,037 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:22:51,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:22:51,300 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:22:51,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:22:51,564 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:22:51,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:22:51,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:22:52,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-07 18:22:52,349 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:22:52,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:22:52,618 [I] DB_PATCHES have already been applied
I20260807 18:22:52.625794 30323 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyv84zy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:22:52.625912 30323 mini_ranger.cc:193] Using host: 127.29.156.212
I20260807 18:22:52.651367 30323 mini_ranger.cc:227] Ranger admin URL: http://127.29.156.212:55139
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:22:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:22:54.059674 30323 mini_ranger.cc:160] Time spent starting Ranger: real 9.691s user 0.014s sys 0.065s
Aug 07, 2026 6:23:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:23:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:23:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:23:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:23:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:23:17.775084 30323 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangerkms_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/rangerkms_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.156.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.156.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:23:18 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:23:18.065114 30323 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:23:18.096982 30323 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:23:19.121690 30323 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:23:19.121874 30323 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms
I20260807 18:23:19.122018 30323 mini_ranger_kms.cc:227] Using postgres at 127.29.156.212:48973
2026-08-07 18:23:19,816 [I] DB FLAVOR :POSTGRES
2026-08-07 18:23:19,816 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:23:19,816 [I] Checking connection
2026-08-07 18:23:19,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:23:20,098 [I] connection success
2026-08-07 18:23:20,098 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:23:20,099 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:23:20,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:23:20,356 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:23:20,356 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:23:20,356 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:23:20,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:48973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:23:20,657 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:23:21.159845 30323 mini_ranger_kms.cc:310] Created kms service
I20260807 18:23:21.369594 30323 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:23:21.416841 30323 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:23:21 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:21.908473 30323 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:23:21.908869 30323 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:23:21.908959 30323 mini_ranger_kms.cc:240] Using host: 127.29.156.212
I20260807 18:23:21.941780 30323 mini_ranger_kms.cc:276] Ranger KMS PID: 13525
I20260807 18:23:21.942011 30323 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.156.212:41035
Aug 07, 2026 6:23:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:23:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:23:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:23:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:23:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:23:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:23:24 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:23:24.758018 30323 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.693s user 0.137s sys 0.528s
I20260807 18:23:24.758343 30323 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:23:24.758608 30323 mini_ranger_kms.cc:401] 127.29.156.212:41035/kms/v1/keys
Aug 07 18:23:24 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
Aug 07 18:23:25 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
Aug 07 18:23:27 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:23:27.896423 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40651
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40651
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:41035/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:23:28.344023 13596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:23:28.344460 13596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:23:28.344712 13596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:23:28.356901 13596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:23:28.357012 13596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:23:28.357101 13596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:23:28.357195 13596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:23:28.357247 13596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:23:28.370097 13596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.156.212:41035/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40651
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:23:28.371490 13596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:23:28.372946 13596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:23:28.387603 13603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:28.387667 13605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:28.387518 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:23:28.393087 13596 server_base.cc:1034] running on GCE node
I20260807 18:23:28.394080 13596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:23:28.395814 13596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:23:28.397156 13596 hybrid_clock.cc:648] HybridClock initialized: now 1786127008397057 us; error 51 us; skew 500 ppm
I20260807 18:23:28.397681 13596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:23:28 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127008, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:23:28.401568 13596 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:23:28.403318 13596 webserver.cc:460] Webserver started at http://127.29.156.254:42197/ using document root <none> and password file <none>
I20260807 18:23:28.404115 13596 fs_manager.cc:359] Metadata directory not provided
I20260807 18:23:28.404289 13596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:23:28.404634 13596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:23:28 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127008, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:28.476730 13596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "dbfcf27c8f8742caa6e5984561666d13"
format_stamp: "Formatted at 2026-08-07 18:23:28 on dist-test-slave-sb2z"
server_key: "958fd21f26d877a413ba327aec300629"
server_key_iv: "f5ff1d1fe998a8b709bea10bcf47352e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:28.477762 13596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "dbfcf27c8f8742caa6e5984561666d13"
format_stamp: "Formatted at 2026-08-07 18:23:28 on dist-test-slave-sb2z"
server_key: "958fd21f26d877a413ba327aec300629"
server_key_iv: "f5ff1d1fe998a8b709bea10bcf47352e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:28.484107 13596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:23:28.505337 13617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:28.507380 13596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:23:28.507649 13596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "dbfcf27c8f8742caa6e5984561666d13"
format_stamp: "Formatted at 2026-08-07 18:23:28 on dist-test-slave-sb2z"
server_key: "958fd21f26d877a413ba327aec300629"
server_key_iv: "f5ff1d1fe998a8b709bea10bcf47352e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:28.508008 13596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:23:28.528937 13596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:23:28.534407 13596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:23:28.534891 13596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:23:28.566430 13691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:40651 every 8 connection(s)
I20260807 18:23:28.566458 13596 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:40651
I20260807 18:23:28.568547 13596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:23:28.570825 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 13596
I20260807 18:23:28.571023 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:23:28.572002 30323 external_mini_cluster.cc:1428] Setting key bfa5f8350cf25d8e39901850c61a2c03
I20260807 18:23:28.574101 13692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:23:28 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:23:28.590924 13692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: Bootstrap starting.
I20260807 18:23:28.595516 13692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: Neither blocks nor log segments found. Creating new log.
I20260807 18:23:28.597047 13692 log.cc:824] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: Log is configured to *not* fsync() on all Append() calls
I20260807 18:23:28.601742 13692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: No bootstrap required, opened a new log
I20260807 18:23:28.602255 13697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:28.574478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.29.156.254:40651)
0807 18:23:28.575670 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:28.575686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:28.575737 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:23:28.578313 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:28.578347 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:28.578927 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:28.580336 (+ 1409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:28.581256 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:28.583810 (+ 2554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:28.585603 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:28.586688 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:28.586893 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:28.592207 (+ 5314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:28.592218 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:28.592239 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:28.592302 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:28.596804 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:28.597926 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:28.597936 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:28.597944 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:28.598125 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:28.598567 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:28.598572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:28.598577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:28.599311 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:28.599743 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:28.600172 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 18:23:28.600643 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":610,"threads_started":1}
I20260807 18:23:28.611891 13692 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } }
I20260807 18:23:28.612476 13692 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:23:28.612604 13692 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfcf27c8f8742caa6e5984561666d13, State: Initialized, Role: FOLLOWER
I20260807 18:23:28.613343 13692 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } }
I20260807 18:23:28.613749 13692 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:23:28.613914 13692 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:23:28.614158 13692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:23:28.616941 13692 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } }
I20260807 18:23:28.617617 13692 leader_election.cc:302] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbfcf27c8f8742caa6e5984561666d13; no voters:
I20260807 18:23:28.618499 13692 leader_election.cc:288] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:23:28.618629 13699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:23:28.621775 13699 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [term 1 LEADER]: Becoming Leader. State: Replica: dbfcf27c8f8742caa6e5984561666d13, State: Running, Role: LEADER
I20260807 18:23:28.622545 13699 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } }
I20260807 18:23:28.623046 13692 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:23:28.625862 13701 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbfcf27c8f8742caa6e5984561666d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } } }
I20260807 18:23:28.626262 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbfcf27c8f8742caa6e5984561666d13. Latest consensus state: current_term: 1 leader_uuid: "dbfcf27c8f8742caa6e5984561666d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfcf27c8f8742caa6e5984561666d13" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40651 } } }
I20260807 18:23:28.626583 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [sys.catalog]: This master's current role is: LEADER
I20260807 18:23:28.626570 13701 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13 [sys.catalog]: This master's current role is: LEADER
I20260807 18:23:28.642117 13709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:23:28.652407 13709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:23:28.653404 13692 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:23:28.682744 13709 catalog_manager.cc:1347] Generated new cluster ID: 969d244954104b569a7494753ad0f756
I20260807 18:23:28.683439 13709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:23:28.719776 13709 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:23:28.722414 13709 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:23:28.734452 13709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: Generated new TSK 0
I20260807 18:23:28.735700 13709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:23:29.531088 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:41035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:23:29 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127009, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:23:30.058674 13760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:23:30.059240 13760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:23:30.059463 13760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:23:30.073742 13760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:23:30.074016 13760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:23:30.074259 13760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:23:30.093218 13760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.156.212:41035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:23:30.095160 13760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:23:30.097182 13760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:23:30.115443 13772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:30.115612 13773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:30.116736 13775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:23:30.116849 13760 server_base.cc:1034] running on GCE node
I20260807 18:23:30.117794 13760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:23:30.118985 13760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:23:30.120308 13760 hybrid_clock.cc:648] HybridClock initialized: now 1786127010120235 us; error 61 us; skew 500 ppm
I20260807 18:23:30.120708 13760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:23:30 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127010, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:23:30.128073 13760 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:23:30.129973 13760 webserver.cc:460] Webserver started at http://127.29.156.193:37241/ using document root <none> and password file <none>
I20260807 18:23:30.130813 13760 fs_manager.cc:359] Metadata directory not provided
I20260807 18:23:30.131052 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:23:30.131562 13760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:23:30 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127010, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:30.152550 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "63231c337680453585ffa190db674edc"
format_stamp: "Formatted at 2026-08-07 18:23:30 on dist-test-slave-sb2z"
server_key: "9c15d56b3aa9579a735c52d8a7753391"
server_key_iv: "f84c111bfb9303fcdc1f9e9ffd227555"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:30.153855 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "63231c337680453585ffa190db674edc"
format_stamp: "Formatted at 2026-08-07 18:23:30 on dist-test-slave-sb2z"
server_key: "9c15d56b3aa9579a735c52d8a7753391"
server_key_iv: "f84c111bfb9303fcdc1f9e9ffd227555"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:30.180980 13760 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.014s sys 0.009s
I20260807 18:23:30.218792 13785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:30.221001 13760 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.006s
I20260807 18:23:30.221354 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "63231c337680453585ffa190db674edc"
format_stamp: "Formatted at 2026-08-07 18:23:30 on dist-test-slave-sb2z"
server_key: "9c15d56b3aa9579a735c52d8a7753391"
server_key_iv: "f84c111bfb9303fcdc1f9e9ffd227555"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:30.221666 13760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:23:30.275396 13760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:23:30.282517 13760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:23:30.282994 13760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:23:30.290759 13760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:23:30.293706 13760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:23:30.293852 13760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:30.294095 13760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:23:30.294195 13760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:23:30 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127009, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:30.733510 13760 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:43773
I20260807 18:23:30.735752 13760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:23:30.736544 13941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:43773 every 8 connection(s)
I20260807 18:23:30.743108 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 13760
I20260807 18:23:30.743387 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:23:30.744486 30323 external_mini_cluster.cc:1428] Setting key b63fff4110837db0597678f28d5f19bb
Aug 07 18:23:30 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127010, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:23:30.792883 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:30.742493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:33545 (local address 127.29.156.254:40651)
0807 18:23:30.764009 (+ 21516us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:30.764016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:30.764038 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:23:30.764107 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:30.764112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:30.764198 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:30.767143 (+ 2945us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:30.767317 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:30.768644 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:30.771573 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:30.772315 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:30.772449 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:30.778415 (+ 5966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:30.778422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:30.778438 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:30.778512 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:30.783220 (+ 4708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:30.784455 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:30.784461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:30.784464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:30.784701 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:30.785454 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:30.785462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:30.785465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:30.785874 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:30.786049 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:30.790756 (+ 4707us) server_negotiation.cc:299] Negotiation successful
0807 18:23:30.791057 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21356,"thread_start_us":21774,"threads_started":1}
I20260807 18:23:30.793253 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:30.743128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40651 (local address 127.29.156.193:33545)
0807 18:23:30.757655 (+ 14527us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:30.757717 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:30.761473 (+ 3756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:30.765063 (+ 3590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:30.765076 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:30.765834 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:23:30.766922 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:30.766952 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:30.768881 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:30.768887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:30.771317 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:30.771335 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:30.772868 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:30.772878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:30.773325 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:30.774289 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:23:30.774322 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:23:30.777950 (+ 3628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:23:30.783517 (+ 5567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:30.783565 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:30.783586 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:30.784242 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:30.784859 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:30.784864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:30.784868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:30.785268 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:30.786158 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:23:30.786167 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:23:30.786675 (+ 508us) client_negotiation.cc:769] Sending connection context
0807 18:23:30.787015 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 18:23:30.788758 (+ 1743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14124,"thread_start_us":16308,"threads_started":1}
I20260807 18:23:30.795387 13942 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40651
I20260807 18:23:30.795801 13942 heartbeater.cc:464] Registering TS with master...
I20260807 18:23:30.796870 13942 heartbeater.cc:511] Master 127.29.156.254:40651 requested a full tablet report, sending...
I20260807 18:23:30.799840 13640 ts_manager.cc:194] Registered new tserver with Master: 63231c337680453585ffa190db674edc (127.29.156.193:43773)
I20260807 18:23:30.803506 13640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:33545
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Aug 07 18:23:30 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127009, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
W20260807 18:23:30.875994 30323 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.156.194: real 0.109s user 0.000s sys 0.026s
I20260807 18:23:30.877028 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:41035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:23:31.426067 13954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:23:31.426371 13954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:23:31.426553 13954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:23:31.434870 13954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:23:31.434993 13954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:23:31.435164 13954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:23:31.446164 13954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.156.212:41035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:23:31.447616 13954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:23:31.449222 13954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:23:31.466624 13970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:31.466609 13971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:31.468632 13977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:23:31.469532 13954 server_base.cc:1034] running on GCE node
I20260807 18:23:31.470264 13954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:23:31.471385 13954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:23:31.472681 13954 hybrid_clock.cc:648] HybridClock initialized: now 1786127011472548 us; error 71 us; skew 500 ppm
I20260807 18:23:31.473132 13954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:23:31 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127011, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:23:31.477193 13954 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:23:31.479133 13954 webserver.cc:460] Webserver started at http://127.29.156.194:46615/ using document root <none> and password file <none>
I20260807 18:23:31.480237 13954 fs_manager.cc:359] Metadata directory not provided
I20260807 18:23:31.480435 13954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:23:31.480894 13954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:23:31 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127011, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:31.499349 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "1b19ab5d2dae4723b23ba8cd0161d043"
format_stamp: "Formatted at 2026-08-07 18:23:31 on dist-test-slave-sb2z"
server_key: "41cde2e1d23914e6f778312757fbf1c7"
server_key_iv: "b8f1057e2d96491332fd6d09a618159e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:31.500463 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "1b19ab5d2dae4723b23ba8cd0161d043"
format_stamp: "Formatted at 2026-08-07 18:23:31 on dist-test-slave-sb2z"
server_key: "41cde2e1d23914e6f778312757fbf1c7"
server_key_iv: "b8f1057e2d96491332fd6d09a618159e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:31.516948 13954 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.010s sys 0.005s
I20260807 18:23:31.544026 13986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:31.546326 13954 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.010s sys 0.001s
I20260807 18:23:31.546559 13954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "1b19ab5d2dae4723b23ba8cd0161d043"
format_stamp: "Formatted at 2026-08-07 18:23:31 on dist-test-slave-sb2z"
server_key: "41cde2e1d23914e6f778312757fbf1c7"
server_key_iv: "b8f1057e2d96491332fd6d09a618159e"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:31.546952 13954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:23:31.573915 13954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:23:31.580005 13954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:23:31.580448 13954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:23:31.582029 13954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:23:31.584271 13954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:23:31.584378 13954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:31.584512 13954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:23:31.584616 13954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:31.658134 13954 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:41201
I20260807 18:23:31.658274 14141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:41201 every 8 connection(s)
I20260807 18:23:31.660068 13954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:23:31.667164 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 13954
I20260807 18:23:31.667366 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:23:31.667896 30323 external_mini_cluster.cc:1428] Setting key 6be7c8cbf8133eccdd521b0d7dd1dbed
Aug 07 18:23:31 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127011, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:23:31.685634 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:31.663368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:47523 (local address 127.29.156.254:40651)
0807 18:23:31.664063 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:31.664068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:31.664880 (+ 812us) server_negotiation.cc:407] Connection header received
0807 18:23:31.667406 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:31.667412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:31.667519 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:31.667729 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:31.669860 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:31.670858 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:31.673687 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:31.674268 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:31.674377 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:31.678962 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:31.678968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:31.678971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:31.679005 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:31.682275 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:31.683112 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:31.683122 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:31.683124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:31.683239 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:31.683754 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:31.683761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:31.683768 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:31.684129 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:31.684291 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:31.685014 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 18:23:31.685242 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":389,"threads_started":1}
I20260807 18:23:31.686650 14145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:31.663895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40651 (local address 127.29.156.194:47523)
0807 18:23:31.664706 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:31.664747 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:31.667097 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:31.668054 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:31.668068 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:31.668772 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:23:31.669597 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:31.669646 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:31.671028 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:31.671036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:31.673478 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:31.673516 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:31.674537 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:31.674547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:31.674989 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:31.676237 (+ 1248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:23:31.676264 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:23:31.678630 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:23:31.682433 (+ 3803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:31.682443 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:31.682466 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:31.682977 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:31.683355 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:31.683360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:31.683364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:31.683634 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:31.684264 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:23:31.684272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:23:31.684620 (+ 348us) client_negotiation.cc:769] Sending connection context
0807 18:23:31.684887 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:23:31.685429 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":349,"threads_started":1}
I20260807 18:23:31.688324 14142 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40651
I20260807 18:23:31.688643 14142 heartbeater.cc:464] Registering TS with master...
I20260807 18:23:31.689466 14142 heartbeater.cc:511] Master 127.29.156.254:40651 requested a full tablet report, sending...
I20260807 18:23:31.691002 13640 ts_manager.cc:194] Registered new tserver with Master: 1b19ab5d2dae4723b23ba8cd0161d043 (127.29.156.194:41201)
I20260807 18:23:31.691877 13640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:47523
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:23:31.739672 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:41035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:23:31.808092 13942 heartbeater.cc:503] Master 127.29.156.254:40651 was elected leader, sending a full tablet report...
W20260807 18:23:32.117738 14151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:23:32.118054 14151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:23:32.118206 14151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:23:32.125815 14151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:23:32.125929 14151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:23:32.126106 14151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:23:32.136332 14151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.156.212:41035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:40651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:23:32.137740 14151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:23:32.139088 14151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:23:32.153501 14158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:32.153538 14159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:23:32.153468 14161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:23:32.154837 14151 server_base.cc:1034] running on GCE node
I20260807 18:23:32.155645 14151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:23:32.156715 14151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:23:32.157960 14151 hybrid_clock.cc:648] HybridClock initialized: now 1786127012157896 us; error 67 us; skew 500 ppm
I20260807 18:23:32.158313 14151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:23:32 dist-test-slave-sb2z krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127012, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:23:32.161819 14151 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:23:32.163604 14151 webserver.cc:460] Webserver started at http://127.29.156.195:44771/ using document root <none> and password file <none>
I20260807 18:23:32.164392 14151 fs_manager.cc:359] Metadata directory not provided
I20260807 18:23:32.164552 14151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:23:32.164913 14151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:23:32 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127012, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:23:32.183596 14151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "690da950a1b3487496f7be415152ee2a"
format_stamp: "Formatted at 2026-08-07 18:23:32 on dist-test-slave-sb2z"
server_key: "2c01a5bbef3893aeb032444da9ac6245"
server_key_iv: "4614c458fcb15cb687d6127305aeb4c5"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:32.184726 14151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "690da950a1b3487496f7be415152ee2a"
format_stamp: "Formatted at 2026-08-07 18:23:32 on dist-test-slave-sb2z"
server_key: "2c01a5bbef3893aeb032444da9ac6245"
server_key_iv: "4614c458fcb15cb687d6127305aeb4c5"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:32.192152 14151 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260807 18:23:32.210549 14172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:32.212604 14151 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:23:32.212802 14151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "690da950a1b3487496f7be415152ee2a"
format_stamp: "Formatted at 2026-08-07 18:23:32 on dist-test-slave-sb2z"
server_key: "2c01a5bbef3893aeb032444da9ac6245"
server_key_iv: "4614c458fcb15cb687d6127305aeb4c5"
server_key_version: "kuduclusterkey@0"
I20260807 18:23:32.213095 14151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:23:32.231469 14151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:23:32.237135 14151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:23:32.237497 14151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:23:32.238759 14151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:23:32.240613 14151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:23:32.240710 14151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:32.240819 14151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:23:32.240895 14151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:23:32.278636 14151 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:38825
I20260807 18:23:32.278680 14329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:38825 every 8 connection(s)
I20260807 18:23:32.280085 14151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:23:32.283053 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 14151
I20260807 18:23:32.283237 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:23:32.283834 30323 external_mini_cluster.cc:1428] Setting key 062b8f91c512b9849a186e678386486f
Aug 07 18:23:32 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127012, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:23:32.301621 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.284666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:60213 (local address 127.29.156.254:40651)
0807 18:23:32.285248 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.285256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.285280 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:23:32.287191 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.287197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.287284 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.287427 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:32.288935 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.289719 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.291284 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.291924 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.292065 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.295593 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:32.295599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:32.295602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:32.295638 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:32.298285 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.299040 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.299047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.299051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.299228 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.299744 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.299750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.299754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.300030 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:32.300213 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.300986 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.301238 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":256,"threads_started":1}
I20260807 18:23:32.302594 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.283767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:40651 (local address 127.29.156.195:60213)
0807 18:23:32.284777 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.284844 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.286919 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.287718 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.287727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.288166 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:23:32.288759 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.288777 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.289888 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.289892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.291130 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.291143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.292181 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.292192 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.292426 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.293085 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:23:32.293113 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:23:32.295335 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:23:32.298441 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:32.298450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:32.298467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:32.298917 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:32.299348 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:23:32.299353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:23:32.299357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:23:32.299629 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:23:32.300177 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:23:32.300185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:23:32.300596 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.300889 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.301433 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":367,"threads_started":1}
I20260807 18:23:32.304214 14332 heartbeater.cc:347] Connected to a master server at 127.29.156.254:40651
I20260807 18:23:32.304572 14332 heartbeater.cc:464] Registering TS with master...
I20260807 18:23:32.305358 14332 heartbeater.cc:511] Master 127.29.156.254:40651 requested a full tablet report, sending...
I20260807 18:23:32.306985 13640 ts_manager.cc:194] Registered new tserver with Master: 690da950a1b3487496f7be415152ee2a (127.29.156.195:38825)
I20260807 18:23:32.307921 13640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:60213
I20260807 18:23:32.312762 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:23:32.331867 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.318146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.29.156.254:40651)
0807 18:23:32.318390 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.318396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.318730 (+ 334us) server_negotiation.cc:407] Connection header received
0807 18:23:32.318860 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.318864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.318943 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.319084 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:32.320737 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.321477 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.323009 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.323433 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.323690 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.326426 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:32.326431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:32.326435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:32.326468 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:32.329252 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.329938 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.329942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.329945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.330063 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.330435 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.330439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.330441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.330743 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:32.330936 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.331269 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.331461 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:23:32.336675 13640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:23:32.344178 13640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:23:32.383179 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.357655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41201 (local address 127.0.0.1:39686)
0807 18:23:32.358909 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.359070 (+ 161us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.359308 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.374873 (+ 15565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.374878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.374939 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:23:32.375606 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.375621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.377530 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.377536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.380012 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.380028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.382150 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.382329 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.382377 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.382444 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.382699 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":61,"spinlock_wait_cycles":19712,"thread_start_us":527,"threads_started":1}
I20260807 18:23:32.383186 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.358990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.29.156.195:38825)
0807 18:23:32.366318 (+ 7328us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.366331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.372648 (+ 6317us) server_negotiation.cc:407] Connection header received
0807 18:23:32.374004 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.374014 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.374329 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.374514 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:23:32.376003 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.378083 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.380229 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.381775 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382002 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.382215 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.382636 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.382807 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7135,"thread_start_us":6968,"threads_started":1}
I20260807 18:23:32.383646 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.357806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.29.156.194:41201)
0807 18:23:32.374213 (+ 16407us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.374222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.374252 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:23:32.374344 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.374353 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.374623 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.374880 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:23:32.375883 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.377383 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.380211 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.381879 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382847 (+ 968us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.383016 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.383086 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.383281 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16244,"thread_start_us":16182,"threads_started":1}
I20260807 18:23:32.383823 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.359549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.29.156.193:43773)
0807 18:23:32.374831 (+ 15282us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.374839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.374866 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:23:32.374960 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.374969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.375213 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.375426 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:23:32.376336 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.378052 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.380781 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.382302 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382510 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.382678 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.383302 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.383479 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15105,"thread_start_us":14930,"threads_started":1}
I20260807 18:23:32.384120 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.358360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38825 (local address 127.0.0.1:58168)
0807 18:23:32.372453 (+ 14093us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.372481 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.373757 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.374857 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.374865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.374921 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:23:32.375761 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.375773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.378250 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.378254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.380048 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.380064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.382002 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382011 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.382240 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.382300 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.382447 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.383677 (+ 1230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13890,"mutex_wait_us":1119,"spinlock_wait_cycles":896,"thread_start_us":15222,"threads_started":1}
I20260807 18:23:32.384539 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.359018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43773 (local address 127.0.0.1:54286)
0807 18:23:32.373333 (+ 14315us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.373352 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.373495 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.375456 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.375473 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.375504 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:23:32.376104 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.376118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.378215 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.378227 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.380584 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.380598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.382536 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.382545 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.382724 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.382760 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.382833 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.382987 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14212,"mutex_wait_us":2,"thread_start_us":13684,"threads_started":1}
I20260807 18:23:32.387804 14050 tablet_service.cc:1467] Processing CreateTablet for tablet 7abf20eb9a344248b7e235c06dc3689b (DEFAULT_TABLE table=test-table [id=bb153e9283494b029469f561e30b0805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:23:32.387921 14235 tablet_service.cc:1467] Processing CreateTablet for tablet 7abf20eb9a344248b7e235c06dc3689b (DEFAULT_TABLE table=test-table [id=bb153e9283494b029469f561e30b0805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:23:32.389174 14050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abf20eb9a344248b7e235c06dc3689b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:23:32.389415 14235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abf20eb9a344248b7e235c06dc3689b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:23:32.389161 13852 tablet_service.cc:1467] Processing CreateTablet for tablet 7abf20eb9a344248b7e235c06dc3689b (DEFAULT_TABLE table=test-table [id=bb153e9283494b029469f561e30b0805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:23:32.390802 13852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abf20eb9a344248b7e235c06dc3689b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:23:32.401536 14367 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Bootstrap starting.
I20260807 18:23:32.405393 14369 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Bootstrap starting.
I20260807 18:23:32.405886 14367 tablet_bootstrap.cc:654] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Neither blocks nor log segments found. Creating new log.
I20260807 18:23:32.407120 14368 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Bootstrap starting.
I20260807 18:23:32.407472 14367 log.cc:824] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Log is configured to *not* fsync() on all Append() calls
I20260807 18:23:32.409267 14369 tablet_bootstrap.cc:654] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Neither blocks nor log segments found. Creating new log.
I20260807 18:23:32.410388 14368 tablet_bootstrap.cc:654] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Neither blocks nor log segments found. Creating new log.
I20260807 18:23:32.410584 14369 log.cc:824] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:23:32.411144 14367 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: No bootstrap required, opened a new log
I20260807 18:23:32.411466 14367 ts_tablet_manager.cc:1397] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 18:23:32.411597 14368 log.cc:824] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:23:32.414505 14369 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: No bootstrap required, opened a new log
I20260807 18:23:32.414935 14369 ts_tablet_manager.cc:1397] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:23:32.421226 14368 tablet_bootstrap.cc:492] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: No bootstrap required, opened a new log
I20260807 18:23:32.421530 14368 ts_tablet_manager.cc:1397] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260807 18:23:32.422856 14367 raft_consensus.cc:348] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.423411 14367 raft_consensus.cc:374] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:23:32.423511 14367 raft_consensus.cc:729] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b19ab5d2dae4723b23ba8cd0161d043, State: Initialized, Role: FOLLOWER
I20260807 18:23:32.424393 14367 consensus_queue.cc:260] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.425742 14142 heartbeater.cc:503] Master 127.29.156.254:40651 was elected leader, sending a full tablet report...
I20260807 18:23:32.426462 14367 ts_tablet_manager.cc:1428] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
I20260807 18:23:32.428092 14369 raft_consensus.cc:348] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.428546 14369 raft_consensus.cc:374] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:23:32.428642 14369 raft_consensus.cc:729] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63231c337680453585ffa190db674edc, State: Initialized, Role: FOLLOWER
I20260807 18:23:32.429283 14369 consensus_queue.cc:260] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.431089 14369 ts_tablet_manager.cc:1428] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Time spent starting tablet: real 0.016s user 0.009s sys 0.004s
I20260807 18:23:32.431375 14368 raft_consensus.cc:348] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.431829 14368 raft_consensus.cc:374] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:23:32.431950 14368 raft_consensus.cc:729] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690da950a1b3487496f7be415152ee2a, State: Initialized, Role: FOLLOWER
I20260807 18:23:32.432694 14368 consensus_queue.cc:260] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.433914 14332 heartbeater.cc:503] Master 127.29.156.254:40651 was elected leader, sending a full tablet report...
I20260807 18:23:32.434652 14368 ts_tablet_manager.cc:1428] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260807 18:23:32.507455 13947 tablet.cc:2369] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:23:32.531994 14333 tablet.cc:2369] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:23:32.565395 14376 raft_consensus.cc:482] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:23:32.565732 14376 raft_consensus.cc:504] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.567854 14376 leader_election.cc:288] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b19ab5d2dae4723b23ba8cd0161d043 (127.29.156.194:41201), 63231c337680453585ffa190db674edc (127.29.156.193:43773)
I20260807 18:23:32.575681 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.568611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:36669 (local address 127.29.156.194:41201)
0807 18:23:32.568878 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.568888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.568915 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:23:32.569008 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.569029 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.569127 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.569218 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:23:32.569923 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.571020 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.573386 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.574603 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.574752 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.574824 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.575216 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.575338 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:23:32.575716 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.568210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41201 (local address 127.29.156.195:36669)
0807 18:23:32.568404 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.568424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.568562 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.569498 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.569504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.569542 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:23:32.569747 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.569767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.571158 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.571163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.573210 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.573226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.574781 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.574787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.575027 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.575068 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.575184 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.575327 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:23:32.576575 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.568640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43773 (local address 127.29.156.195:56089)
0807 18:23:32.569430 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 18:23:32.569450 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:23:32.569570 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:23:32.570156 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:23:32.570162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:23:32.570188 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:23:32.570636 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.570650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.572140 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.572152 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.573983 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:23:32.573996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.575648 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.575661 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:23:32.575891 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.575928 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:23:32.576082 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:23:32.576221 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":410,"threads_started":1}
I20260807 18:23:32.576841 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.569635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:56089 (local address 127.29.156.193:43773)
0807 18:23:32.569831 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.569837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.569865 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:23:32.569935 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.569940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.570011 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.570118 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:23:32.570825 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.571918 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.574159 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.575248 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.575385 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.575454 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.576196 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.576333 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:23:32.577335 14079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abf20eb9a344248b7e235c06dc3689b" candidate_uuid: "690da950a1b3487496f7be415152ee2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" is_pre_election: true
I20260807 18:23:32.578047 14079 raft_consensus.cc:2454] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690da950a1b3487496f7be415152ee2a in term 0.
I20260807 18:23:32.578305 13876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abf20eb9a344248b7e235c06dc3689b" candidate_uuid: "690da950a1b3487496f7be415152ee2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63231c337680453585ffa190db674edc" is_pre_election: true
I20260807 18:23:32.578929 14173 leader_election.cc:302] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b19ab5d2dae4723b23ba8cd0161d043, 690da950a1b3487496f7be415152ee2a; no voters:
I20260807 18:23:32.578980 13876 raft_consensus.cc:2454] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690da950a1b3487496f7be415152ee2a in term 0.
I20260807 18:23:32.579355 14376 raft_consensus.cc:2781] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:23:32.579545 14376 raft_consensus.cc:482] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:23:32.579761 14376 raft_consensus.cc:3037] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:23:32.582406 14376 raft_consensus.cc:504] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.583086 14376 leader_election.cc:288] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [CANDIDATE]: Term 1 election: Requested vote from peers 1b19ab5d2dae4723b23ba8cd0161d043 (127.29.156.194:41201), 63231c337680453585ffa190db674edc (127.29.156.193:43773)
I20260807 18:23:32.583609 14079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abf20eb9a344248b7e235c06dc3689b" candidate_uuid: "690da950a1b3487496f7be415152ee2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b19ab5d2dae4723b23ba8cd0161d043"
I20260807 18:23:32.583794 13876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abf20eb9a344248b7e235c06dc3689b" candidate_uuid: "690da950a1b3487496f7be415152ee2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63231c337680453585ffa190db674edc"
I20260807 18:23:32.583930 14079 raft_consensus.cc:3037] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:23:32.583994 13876 raft_consensus.cc:3037] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:23:32.586417 13876 raft_consensus.cc:2454] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690da950a1b3487496f7be415152ee2a in term 1.
I20260807 18:23:32.586957 14173 leader_election.cc:302] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63231c337680453585ffa190db674edc, 690da950a1b3487496f7be415152ee2a; no voters:
I20260807 18:23:32.587304 14376 raft_consensus.cc:2781] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:23:32.587564 14079 raft_consensus.cc:2454] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690da950a1b3487496f7be415152ee2a in term 1.
I20260807 18:23:32.588204 14376 raft_consensus.cc:686] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 1 LEADER]: Becoming Leader. State: Replica: 690da950a1b3487496f7be415152ee2a, State: Running, Role: LEADER
I20260807 18:23:32.588785 14376 consensus_queue.cc:237] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } }
I20260807 18:23:32.594206 13640 catalog_manager.cc:5495] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 690da950a1b3487496f7be415152ee2a (127.29.156.195). New cstate: current_term: 1 leader_uuid: "690da950a1b3487496f7be415152ee2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "690da950a1b3487496f7be415152ee2a" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38825 } health_report { overall_health: HEALTHY } } }
I20260807 18:23:32.626955 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.620318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.29.156.195:38825)
0807 18:23:32.620513 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.620519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.620775 (+ 256us) server_negotiation.cc:407] Connection header received
0807 18:23:32.621032 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.621037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.621186 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.621324 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:23:32.621827 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.622667 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.624336 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.624944 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.625060 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.625733 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:23:32.626297 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:23:32.626437 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.626544 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.626615 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:23:32.637164 13876 raft_consensus.cc:1261] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 1 FOLLOWER]: Refusing update from remote peer 690da950a1b3487496f7be415152ee2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:23:32.637247 14079 raft_consensus.cc:1261] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 1 FOLLOWER]: Refusing update from remote peer 690da950a1b3487496f7be415152ee2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:23:32.638290 14376 consensus_queue.cc:1035] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b19ab5d2dae4723b23ba8cd0161d043" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:23:32.638654 14379 consensus_queue.cc:1035] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "63231c337680453585ffa190db674edc" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:23:32.655516 14384 mvcc.cc:205] Tried to move back new op lower bound from 7315976243749113856 to 7315976243567595520. Current Snapshot: MvccSnapshot[applied={T|T < 7315976243749113856}]
I20260807 18:23:32.684033 13639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:23:32.684638 13639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:23:32.689706 13639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbfcf27c8f8742caa6e5984561666d13: Sending DeleteTablet for 3 replicas of tablet 7abf20eb9a344248b7e235c06dc3689b
I20260807 18:23:32.691473 14235 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abf20eb9a344248b7e235c06dc3689b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:23:32 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:58168
I20260807 18:23:32.691434 13852 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abf20eb9a344248b7e235c06dc3689b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:23:32 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:54286
I20260807 18:23:32.692494 14050 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abf20eb9a344248b7e235c06dc3689b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:23:32 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39686
I20260807 18:23:32.693778 14399 tablet_replica.cc:330] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: stopping tablet replica
I20260807 18:23:32.694304 14399 raft_consensus.cc:2229] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:23:32.695201 14399 raft_consensus.cc:2258] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:23:32.695670 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 13760
I20260807 18:23:32.698969 14399 ts_tablet_manager.cc:1905] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:23:32.702020 14398 tablet_replica.cc:330] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: stopping tablet replica
I20260807 18:23:32.702486 14398 raft_consensus.cc:2229] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:23:32.702982 14398 raft_consensus.cc:2258] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:23:32.709214 14400 tablet_replica.cc:330] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: stopping tablet replica
I20260807 18:23:32.709640 14398 ts_tablet_manager.cc:1905] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:23:32.709723 14400 raft_consensus.cc:2229] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:23:32.710301 14400 raft_consensus.cc:2258] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:23:32.712460 14400 ts_tablet_manager.cc:1905] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:23:32.715055 14399 ts_tablet_manager.cc:1918] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:23:32.715287 14399 log.cc:1196] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/7abf20eb9a344248b7e235c06dc3689b
I20260807 18:23:32.716005 14399 ts_tablet_manager.cc:1939] T 7abf20eb9a344248b7e235c06dc3689b P 690da950a1b3487496f7be415152ee2a: Deleting consensus metadata
I20260807 18:23:32.718201 13624 catalog_manager.cc:4841] TS 690da950a1b3487496f7be415152ee2a (127.29.156.195:38825): tablet 7abf20eb9a344248b7e235c06dc3689b (table test-table [id=bb153e9283494b029469f561e30b0805]) successfully deleted
Aug 07 18:23:32 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:23:32.721553 14400 ts_tablet_manager.cc:1918] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:23:32.721840 14400 log.cc:1196] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/7abf20eb9a344248b7e235c06dc3689b
I20260807 18:23:32.722550 14400 ts_tablet_manager.cc:1939] T 7abf20eb9a344248b7e235c06dc3689b P 1b19ab5d2dae4723b23ba8cd0161d043: Deleting consensus metadata
I20260807 18:23:32.724789 13622 catalog_manager.cc:4841] TS 1b19ab5d2dae4723b23ba8cd0161d043 (127.29.156.194:41201): tablet 7abf20eb9a344248b7e235c06dc3689b (table test-table [id=bb153e9283494b029469f561e30b0805]) successfully deleted
I20260807 18:23:32.725893 14398 ts_tablet_manager.cc:1918] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:23:32.726071 14398 log.cc:1196] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/7abf20eb9a344248b7e235c06dc3689b
I20260807 18:23:32.726843 14398 ts_tablet_manager.cc:1939] T 7abf20eb9a344248b7e235c06dc3689b P 63231c337680453585ffa190db674edc: Deleting consensus metadata
I20260807 18:23:32.728996 13622 catalog_manager.cc:4841] TS 63231c337680453585ffa190db674edc (127.29.156.193:43773): tablet 7abf20eb9a344248b7e235c06dc3689b (table test-table [id=bb153e9283494b029469f561e30b0805]) successfully deleted
I20260807 18:23:32.731952 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.696950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.29.156.193:43773)
0807 18:23:32.697148 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.697154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.707959 (+ 10805us) server_negotiation.cc:407] Connection header received
0807 18:23:32.708599 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.708608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.708725 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.708809 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:32.711181 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.712401 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.714771 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.715576 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.715700 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.722388 (+ 6688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:32.722406 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:32.722426 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:32.722482 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:32.727816 (+ 5334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.728975 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.728986 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.728990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.729171 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.729762 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.729767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.729771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.730334 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:32.730719 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.731179 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.731442 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:23:32.732980 13933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:23:32.855962 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 13760
I20260807 18:23:32.883082 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 13954
Aug 07 18:23:32 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:23:32.902663 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:32.883955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.29.156.194:41201)
0807 18:23:32.884223 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:32.884230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:32.884258 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:23:32.884506 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:32.884513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:32.884610 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:32.884740 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:32.886482 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.887799 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.889209 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:32.889872 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:32.890040 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:32.895368 (+ 5328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:32.895389 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:32.895411 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:32.895453 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:32.898760 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.899690 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.899699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.899705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.899847 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:32.900323 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:32.900330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:32.900335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:32.901004 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:32.901384 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:32.902032 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 18:23:32.902272 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 18:23:32.903442 14135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:23:32.998977 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 13954
I20260807 18:23:33.026079 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 14151
Aug 07 18:23:33 dist-test-slave-sb2z krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:23:33.046350 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:23:33.027166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.29.156.195:38825)
0807 18:23:33.027355 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:23:33.027360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:23:33.027380 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:23:33.027481 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:23:33.027485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:23:33.027619 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:23:33.027704 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:23:33.029676 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:33.030540 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:33.032669 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:23:33.033275 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:23:33.033409 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:23:33.039440 (+ 6031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:23:33.039456 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:23:33.039473 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:23:33.039516 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:23:33.043067 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:33.044021 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:33.044030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:33.044032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:33.044191 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:23:33.044666 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:23:33.044670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:23:33.044672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:23:33.045040 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:23:33.045400 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 18:23:33.045696 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:23:33.045959 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:23:33.047142 14323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:23:33.172492 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 14151
I20260807 18:23:33.202355 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 13596
I20260807 18:23:33.203773 13683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:23:33.324025 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 13596
I20260807 18:23:33.356287 30323 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:23:33.705317 30323 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:23:33.705627 30323 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:23:34.061803 30323 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:23:34.062 UTC [13047] LOG: received smart shutdown request
2026-08-07 18:23:34.067 UTC [13047] LOG: background worker "logical replication launcher" (PID 13055) exited with exit code 1
2026-08-07 18:23:34.067 UTC [13050] LOG: shutting down
2026-08-07 18:23:34.281 UTC [13047] LOG: database system is shut down
2026-08-07T18:23:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56083 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:23:34 dist-test-slave-sb2z krb5kdc[14421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:23:34 dist-test-slave-sb2z krb5kdc[14421](info): set up 2 sockets
Aug 07 18:23:34 dist-test-slave-sb2z krb5kdc[14421](info): commencing operation
krb5kdc: starting...
W20260807 18:23:35.758348 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:23:35 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:23:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangeradmin_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangeradmin_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangerlookup_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangerlookup_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
I20260807 18:23:36.215467 30323 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyv84zy/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/postgres -l logfile start
2026-08-07 18:23:39.073 UTC [14468] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:23:39.073 UTC [14468] LOG: listening on IPv4 address "127.29.156.212", port 47641
2026-08-07 18:23:39.077 UTC [14468] LOG: listening on Unix socket "/tmp/.s.PGSQL.47641"
2026-08-07 18:23:39.100 UTC [14471] LOG: database system was shut down at 2026-08-07 18:23:37 UTC
2026-08-07 18:23:39.104 UTC [14468] LOG: database system is ready to accept connections
I20260807 18:23:40.282763 30323 mini_postgres.cc:96] Postgres bound to 47641
2026-08-07 18:23:40.313 UTC [14486] FATAL: database "slave" does not exist
127.29.156.212:47641 - accepting connections
I20260807 18:23:40.313982 30323 mini_ranger.cc:161] Starting Ranger...
I20260807 18:23:40.361241 30323 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:23:41.000268 30323 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:23:41.000396 30323 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin
2026-08-07 18:23:41,320 [I] DB FLAVOR :POSTGRES
2026-08-07 18:23:41,321 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:23:41,321 [I] Checking connection..
2026-08-07 18:23:41,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:23:41,574 [I] Checking connection passed.
2026-08-07 18:23:41,574 [I] --------- Verifying version history table ---------
2026-08-07 18:23:41,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:23:41,850 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:23:41,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:23:42,122 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:23:42,123 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:23:42,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:23:42.399 UTC [14585] WARNING: there is no transaction in progress
2026-08-07 18:23:42,408 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:23:42,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:23:42,683 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:23:42,683 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:23:42,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:23:42,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:23:43,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-07 18:23:43,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:23:43,799 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:23:43,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:23:44,067 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:23:44,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:23:44,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:23:44,623 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:23:44,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:23:45.725 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.745 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.764 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.781 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.798 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.822 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.885 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.893 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.910 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.922 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.933 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.950 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.962 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.971 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.980 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:45.990 UTC [14745] WARNING: there is no transaction in progress
2026-08-07 18:23:46,757 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:23:46,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:23:47,011 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:23:47,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:23:47,281 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:23:47,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:23:47,553 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:23:47,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:23:47,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:23:48,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-07 18:23:48,370 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:23:48,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:23:48,651 [I] DB_PATCHES have already been applied
I20260807 18:23:48.657761 30323 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyv84zy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:23:48.657857 30323 mini_ranger.cc:193] Using host: 127.29.156.212
I20260807 18:23:48.681649 30323 mini_ranger.cc:227] Ranger admin URL: http://127.29.156.212:55281
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:23:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:23:50.077607 30323 mini_ranger.cc:160] Time spent starting Ranger: real 9.764s user 0.001s sys 0.074s
Aug 07, 2026 6:24:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:24:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:24:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:24:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:24:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:24:13.641495 30323 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.156.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.156.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangerkms_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.156.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/rangerkms_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.156.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.156.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.156.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/HTTP_127.29.156.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:24:13 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127053, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:24:13.865082 30323 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:24:13.894361 30323 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:24:14.886061 30323 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:24:14.886209 30323 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms
I20260807 18:24:14.886286 30323 mini_ranger_kms.cc:227] Using postgres at 127.29.156.212:47641
2026-08-07 18:24:15,303 [I] DB FLAVOR :POSTGRES
2026-08-07 18:24:15,303 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:24:15,303 [I] Checking connection
2026-08-07 18:24:15,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:24:15,603 [I] connection success
2026-08-07 18:24:15,603 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:24:15,603 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:24:15,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:24:15,850 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:24:15,850 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:24:15,851 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:24:15,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.156.212:47641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:24:16,123 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:24:16.640057 30323 mini_ranger_kms.cc:310] Created kms service
I20260807 18:24:16.845193 30323 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:24:16.890295 30323 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:24:16 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127053, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:17.275418 30323 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:24:17.275807 30323 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyv84zy/build/asan/bin/postgresql.jar:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyv84zy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:24:17.275888 30323 mini_ranger_kms.cc:240] Using host: 127.29.156.212
I20260807 18:24:17.298796 30323 mini_ranger_kms.cc:276] Ranger KMS PID: 15003
I20260807 18:24:17.298947 30323 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.156.212:50475
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:24:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:24:19 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127059, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:20.036707 30323 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.172s user 0.084s sys 0.370s
I20260807 18:24:20.036976 30323 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:24:20.037179 30323 mini_ranger_kms.cc:401] 127.29.156.212:50475/kms/v1/keys
Aug 07 18:24:20 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127059, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
Aug 07 18:24:20 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127059, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.156.212@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
Aug 07 18:24:23 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 18:24:23.376425 30323 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.108s user 0.002s sys 0.054s
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:24:23.452414 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37073
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37073
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:23.818913 15075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:23.819298 15075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:23.819458 15075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:23.826973 15075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:23.827139 15075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:23.827243 15075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:23.827344 15075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:23.827431 15075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:23.838523 15075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.156.212:50475/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37073
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:23.839958 15075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:23.841351 15075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:23.854995 15085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:23.854913 15081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:23.855473 15080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:23.855820 15075 server_base.cc:1034] running on GCE node
I20260807 18:24:23.856685 15075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:23.858218 15075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:23.859447 15075 hybrid_clock.cc:648] HybridClock initialized: now 1786127063859390 us; error 68 us; skew 500 ppm
I20260807 18:24:23.859930 15075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:23 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:23.863456 15075 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:23.865523 15075 webserver.cc:460] Webserver started at http://127.29.156.254:45111/ using document root <none> and password file <none>
I20260807 18:24:23.866263 15075 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:23.866421 15075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:23.866792 15075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:24:23 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:23.938535 15075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "3e335caee34040939571abb57b01f419"
format_stamp: "Formatted at 2026-08-07 18:24:23 on dist-test-slave-sb2z"
server_key: "ea2059f6404ae580f71f3e9ddf1eb3c7"
server_key_iv: "30835637911daf34f8320eba31557e79"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:23.939455 15075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "3e335caee34040939571abb57b01f419"
format_stamp: "Formatted at 2026-08-07 18:24:23 on dist-test-slave-sb2z"
server_key: "ea2059f6404ae580f71f3e9ddf1eb3c7"
server_key_iv: "30835637911daf34f8320eba31557e79"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:23.945169 15075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:24:23.965092 15096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:23.966730 15075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:24:23.966935 15075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "3e335caee34040939571abb57b01f419"
format_stamp: "Formatted at 2026-08-07 18:24:23 on dist-test-slave-sb2z"
server_key: "ea2059f6404ae580f71f3e9ddf1eb3c7"
server_key_iv: "30835637911daf34f8320eba31557e79"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:23.967175 15075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:23.984179 15075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:23.988858 15075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:23.989260 15075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:24.012588 15171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37073 every 8 connection(s)
I20260807 18:24:24.012604 15075 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37073
I20260807 18:24:24.014508 15075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:24.018862 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15075
I20260807 18:24:24.019134 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:24:24.019378 15172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:24.020125 30323 external_mini_cluster.cc:1428] Setting key c00a73dc6a60cfaadd3514b7f53499ed
I20260807 18:24:24.034474 15172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Bootstrap starting.
Aug 07 18:24:24 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:24.038250 15172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:24.039527 15172 log.cc:824] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:24.042951 15172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: No bootstrap required, opened a new log
I20260807 18:24:24.045945 15177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:24.022489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.29.156.254:37073)
0807 18:24:24.023578 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:24.023595 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:24.023693 (+ 98us) server_negotiation.cc:407] Connection header received
0807 18:24:24.026165 (+ 2472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:24.026190 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:24.026654 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:24.027014 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:24.028730 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:24.030103 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:24.032098 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:24.032837 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:24.032981 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:24.037704 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:24.037712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:24.037728 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:24.037774 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:24.041481 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:24.042347 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:24.042354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:24.042361 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:24.042537 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:24.042967 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:24.042971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:24.042973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:24.043652 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:24.043970 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:24.044441 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 18:24:24.044897 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":596,"threads_started":1}
I20260807 18:24:24.051934 15172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:24.052450 15172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:24.052584 15172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e335caee34040939571abb57b01f419, State: Initialized, Role: FOLLOWER
I20260807 18:24:24.053238 15172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:24.053472 15172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:24.053638 15172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:24.053805 15172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:24.056111 15172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:24.056619 15172 leader_election.cc:302] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e335caee34040939571abb57b01f419; no voters:
I20260807 18:24:24.057286 15172 leader_election.cc:288] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:24:24.057462 15182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:24.058351 15182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 LEADER]: Becoming Leader. State: Replica: 3e335caee34040939571abb57b01f419, State: Running, Role: LEADER
I20260807 18:24:24.058940 15182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:24.059811 15172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:24.062331 15186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e335caee34040939571abb57b01f419. Latest consensus state: current_term: 1 leader_uuid: "3e335caee34040939571abb57b01f419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } } }
I20260807 18:24:24.062803 15186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:24.062358 15184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e335caee34040939571abb57b01f419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } } }
I20260807 18:24:24.063300 15184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:24.064399 15193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:24.069535 15193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:24.072646 15172 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:24:24.091816 15193 catalog_manager.cc:1347] Generated new cluster ID: d87bbd7bb9404484aef6853917b8521e
I20260807 18:24:24.092036 15193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:24.107394 15193 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:24:24.109508 15193 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:24.124413 15193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Generated new TSK 0
I20260807 18:24:24.125569 15193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:24:24.895958 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:25 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:24:25.340613 15244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:25.341193 15244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:25.341428 15244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:25.353502 15244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:25.353675 15244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:25.353935 15244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:24:25.370137 15244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:25.372016 15244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:25.373863 15244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:25.392347 15256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:25.393014 15257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:25.399101 15260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:25.400228 15244 server_base.cc:1034] running on GCE node
I20260807 18:24:25.401085 15244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:25.402349 15244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:25.403645 15244 hybrid_clock.cc:648] HybridClock initialized: now 1786127065403576 us; error 52 us; skew 500 ppm
I20260807 18:24:25.404204 15244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:25 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:25.408763 15244 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:24:25.413266 15244 webserver.cc:460] Webserver started at http://127.29.156.193:34829/ using document root <none> and password file <none>
I20260807 18:24:25.414374 15244 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:25.414629 15244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:25.415050 15244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:24:25 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:25.439664 15244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "2c04216c7dc746de8d68405e561da07d"
format_stamp: "Formatted at 2026-08-07 18:24:25 on dist-test-slave-sb2z"
server_key: "5f7efd40b0aad8dc43faf6773c0799ef"
server_key_iv: "6e20b4a13a538973dc1df62b21ed09a0"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:25.441099 15244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "2c04216c7dc746de8d68405e561da07d"
format_stamp: "Formatted at 2026-08-07 18:24:25 on dist-test-slave-sb2z"
server_key: "5f7efd40b0aad8dc43faf6773c0799ef"
server_key_iv: "6e20b4a13a538973dc1df62b21ed09a0"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:25.455091 15244 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.007s sys 0.004s
I20260807 18:24:25.490681 15272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:25.492818 15244 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260807 18:24:25.493121 15244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "2c04216c7dc746de8d68405e561da07d"
format_stamp: "Formatted at 2026-08-07 18:24:25 on dist-test-slave-sb2z"
server_key: "5f7efd40b0aad8dc43faf6773c0799ef"
server_key_iv: "6e20b4a13a538973dc1df62b21ed09a0"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:25.493613 15244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:25.575994 15244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:25.580459 15244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:25.580798 15244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:25.581947 15244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:25.583689 15244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:25.583803 15244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:25.583954 15244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:25.584064 15244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:25.659112 15244 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:41013
I20260807 18:24:25.659116 15437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:41013 every 8 connection(s)
I20260807 18:24:25.660871 15244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:24:25.670881 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15244
I20260807 18:24:25.671169 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:24:25.672569 30323 external_mini_cluster.cc:1428] Setting key 7554d76a9a80f2f669d0dc5d162db3c5
Aug 07 18:24:25 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
Aug 07 18:24:25 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:25.723686 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:25.666637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:57045 (local address 127.29.156.254:37073)
0807 18:24:25.697161 (+ 30524us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:25.697170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:25.698460 (+ 1290us) server_negotiation.cc:407] Connection header received
0807 18:24:25.701327 (+ 2867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:25.701336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:25.701455 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:25.701551 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:25.704046 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:25.705328 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:25.708042 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:25.708807 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:25.708925 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:25.714068 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:25.714106 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:25.714115 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:25.714198 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:25.718407 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:25.720315 (+ 1908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:25.720323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:25.720328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:25.720525 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:25.721198 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:25.721204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:25.721207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:25.721635 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:25.721782 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:25.722932 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0807 18:24:25.723200 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30302,"thread_start_us":31824,"threads_started":1}
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
I20260807 18:24:25.726137 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:25.667397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.193:57045)
0807 18:24:25.697826 (+ 30429us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:25.697897 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:25.700960 (+ 3063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:25.702208 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:25.702221 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:25.702976 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:25.703853 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:25.703875 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:25.705544 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:25.705553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:25.707863 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:25.707883 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:25.709305 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:25.709314 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:25.709658 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:25.710478 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:25.710525 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:25.713745 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:25.719352 (+ 5607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:25.719366 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:25.719388 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:25.720147 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:25.720707 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:25.720714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:25.720719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:25.721048 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:25.721884 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:25.721896 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:25.722420 (+ 524us) client_negotiation.cc:769] Sending connection context
0807 18:24:25.724312 (+ 1892us) client_negotiation.cc:240] Negotiation successful
0807 18:24:25.724753 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30097,"thread_start_us":29785,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
I20260807 18:24:25.728163 15439 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:25.728498 15439 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:25.729295 15439 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:25.731820 15116 ts_manager.cc:194] Registered new tserver with Master: 2c04216c7dc746de8d68405e561da07d (127.29.156.193:41013)
I20260807 18:24:25.734056 15116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:57045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
W20260807 18:24:25.776695 30323 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.156.194: real 0.101s user 0.003s sys 0.025s
I20260807 18:24:25.777722 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:26 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127065, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
W20260807 18:24:26.287539 15447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:26.288080 15447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:26.288276 15447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:26.300024 15447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:26.300274 15447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:26.300500 15447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:24:26.316406 15447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:26.318411 15447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:26.320252 15447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:26.360174 15456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:26.361253 15455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:26.368021 15447 server_base.cc:1034] running on GCE node
W20260807 18:24:26.369143 15463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:26.370414 15447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:26.380077 15447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:26.383577 15447 hybrid_clock.cc:648] HybridClock initialized: now 1786127066383425 us; error 151 us; skew 500 ppm
I20260807 18:24:26.384130 15447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:26 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:26.388851 15447 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:24:26.391284 15447 webserver.cc:460] Webserver started at http://127.29.156.194:43777/ using document root <none> and password file <none>
I20260807 18:24:26.392436 15447 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:26.392589 15447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:26.393081 15447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:24:26 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:26.411187 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "79f0c48de41c43ff8f274cb93b0ef99d"
format_stamp: "Formatted at 2026-08-07 18:24:26 on dist-test-slave-sb2z"
server_key: "d3c776dd53ffea4a71c665989327aaed"
server_key_iv: "4ffcdace3b848cd3e7aeae8a8e1fed04"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:26.412230 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "79f0c48de41c43ff8f274cb93b0ef99d"
format_stamp: "Formatted at 2026-08-07 18:24:26 on dist-test-slave-sb2z"
server_key: "d3c776dd53ffea4a71c665989327aaed"
server_key_iv: "4ffcdace3b848cd3e7aeae8a8e1fed04"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:26.420414 15447 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260807 18:24:26.442224 15484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:26.444082 15447 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.004s
I20260807 18:24:26.444290 15447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "79f0c48de41c43ff8f274cb93b0ef99d"
format_stamp: "Formatted at 2026-08-07 18:24:26 on dist-test-slave-sb2z"
server_key: "d3c776dd53ffea4a71c665989327aaed"
server_key_iv: "4ffcdace3b848cd3e7aeae8a8e1fed04"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:26.444553 15447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:26.467715 15447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:26.472247 15447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:26.472579 15447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:26.473750 15447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:26.475371 15447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:26.475502 15447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:26.475698 15447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:26.475776 15447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:26.520576 15447 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:35975
I20260807 18:24:26.520603 15657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:35975 every 8 connection(s)
I20260807 18:24:26.522044 15447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:24:26.524170 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15447
I20260807 18:24:26.524380 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:24:26.525233 30323 external_mini_cluster.cc:1428] Setting key f9ed5cf779d5c0605bec4fb2b90d80c7
Aug 07 18:24:26 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127066, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
I20260807 18:24:26.574539 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:26.528618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:55853 (local address 127.29.156.254:37073)
0807 18:24:26.549303 (+ 20685us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:26.549312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:26.550480 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 18:24:26.553402 (+ 2922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:26.553408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:26.553505 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:26.553619 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:26.557780 (+ 4161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:26.558897 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:26.561406 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:26.562046 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:26.562146 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:26.566879 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:26.566886 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:26.566890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:26.566933 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:26.570564 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:26.571600 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:26.571606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:26.571609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:26.571764 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:26.572320 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:26.572324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:26.572328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:26.572732 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:26.572867 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:26.573872 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0807 18:24:26.574157 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20509,"thread_start_us":21665,"threads_started":1}
I20260807 18:24:26.576778 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:26.529620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.194:55853)
0807 18:24:26.550056 (+ 20436us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:26.550117 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:26.552904 (+ 2787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:26.555998 (+ 3094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:26.556012 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:26.556686 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:26.557545 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:26.557571 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:26.559126 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:26.559130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:26.561184 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:26.561199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:26.562488 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:26.562494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:26.562785 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:26.563708 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:26.563740 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:26.566599 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:26.570753 (+ 4154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:26.570763 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:26.570783 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:26.571414 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:26.571917 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:26.571922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:26.571925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:26.572177 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:26.572961 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:26.572970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:26.573408 (+ 438us) client_negotiation.cc:769] Sending connection context
0807 18:24:26.574767 (+ 1359us) client_negotiation.cc:240] Negotiation successful
0807 18:24:26.575320 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20026,"thread_start_us":19736,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
I20260807 18:24:26.578372 15658 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:26.578714 15658 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:26.579640 15658 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:26.583209 15116 ts_manager.cc:194] Registered new tserver with Master: 79f0c48de41c43ff8f274cb93b0ef99d (127.29.156.194:35975)
I20260807 18:24:26.584380 15116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:55853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:24:26.623935 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:26.738309 15439 heartbeater.cc:503] Master 127.29.156.254:37073 was elected leader, sending a full tablet report...
W20260807 18:24:27.004738 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:27.005105 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:27.005276 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:27.013247 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:27.013386 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:27.013562 15669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:24:27.027647 15669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:27.029165 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:27.030537 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:27.044482 15676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:27.044515 15677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:27.044518 15679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:27.045831 15669 server_base.cc:1034] running on GCE node
I20260807 18:24:27.046497 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:27.047511 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:27.048785 15669 hybrid_clock.cc:648] HybridClock initialized: now 1786127067048738 us; error 42 us; skew 500 ppm
I20260807 18:24:27.049171 15669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127067, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:27.052734 15669 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:24:27.054301 15669 webserver.cc:460] Webserver started at http://127.29.156.195:39607/ using document root <none> and password file <none>
I20260807 18:24:27.055020 15669 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:27.055140 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:27.055452 15669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127067, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:27.074414 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "173d1fb964ad4703b0881119dace43e7"
format_stamp: "Formatted at 2026-08-07 18:24:27 on dist-test-slave-sb2z"
server_key: "86bf5771943ac7278e35cad070e951f2"
server_key_iv: "30eb04cb1f1288ff8354ad47c0fa52ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:27.075656 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "173d1fb964ad4703b0881119dace43e7"
format_stamp: "Formatted at 2026-08-07 18:24:27 on dist-test-slave-sb2z"
server_key: "86bf5771943ac7278e35cad070e951f2"
server_key_iv: "30eb04cb1f1288ff8354ad47c0fa52ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:27.083338 15669 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:24:27.103324 15688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:27.105371 15669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:24:27.105613 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "173d1fb964ad4703b0881119dace43e7"
format_stamp: "Formatted at 2026-08-07 18:24:27 on dist-test-slave-sb2z"
server_key: "86bf5771943ac7278e35cad070e951f2"
server_key_iv: "30eb04cb1f1288ff8354ad47c0fa52ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:27.105921 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:27.124642 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:27.129101 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:27.129365 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:27.130357 15669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:27.131811 15669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:27.131888 15669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:27.132021 15669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:27.132103 15669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:27.169404 15669 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:38973
I20260807 18:24:27.169451 15857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:38973 every 8 connection(s)
I20260807 18:24:27.171132 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:24:27.173110 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15669
I20260807 18:24:27.173287 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:24:27.173748 30323 external_mini_cluster.cc:1428] Setting key ac957d5bbe10ed0da41fe0fa5ac37bd8
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127067, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:27.193097 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:27.174050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:46427 (local address 127.29.156.254:37073)
0807 18:24:27.174889 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:27.174894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:27.175826 (+ 932us) server_negotiation.cc:407] Connection header received
0807 18:24:27.178055 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:27.178060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:27.178146 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:27.178252 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:27.179759 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.180497 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.182112 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.182558 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.182660 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:27.186457 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:27.186462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:27.186466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:27.186500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:27.189807 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.190600 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.190611 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.190613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.190740 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.191237 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.191241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.191243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.191601 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:27.191762 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:27.192478 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 18:24:27.192725 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":262,"threads_started":1}
I20260807 18:24:27.194065 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:27.174740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.195:46427)
0807 18:24:27.175670 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:27.175709 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:27.177791 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:27.178466 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:27.178477 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:27.178887 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:27.179571 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:27.179599 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.180662 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.180666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:27.181943 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:27.181955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.182804 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.182809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:27.183079 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:27.183928 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:27.183953 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:27.186206 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:27.189971 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:27.189980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:27.190007 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:27.190464 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:27.190858 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:27.190864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:27.190867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:27.191103 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:27.191745 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:27.191753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:27.192100 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 18:24:27.192374 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 18:24:27.192928 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":398,"threads_started":1}
I20260807 18:24:27.195489 15858 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:27.195888 15858 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:27.196611 15858 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:27.198216 15116 ts_manager.cc:194] Registered new tserver with Master: 173d1fb964ad4703b0881119dace43e7 (127.29.156.195:38973)
I20260807 18:24:27.199119 15116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:46427
I20260807 18:24:27.202529 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:24:27.204373 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15244
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:24:27.222873 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:27.205377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.29.156.193:41013)
0807 18:24:27.206229 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:27.206236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:27.206262 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:24:27.206319 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:27.206325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:27.206537 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:27.206733 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:27.208434 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.209474 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.211125 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.211753 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.211860 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:27.216543 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:27.216556 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:27.216570 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:27.216617 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:27.219650 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.220472 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.220480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.220482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.220660 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.221112 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.221117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.221120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.221491 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:27.221810 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:27.222208 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 18:24:27.222518 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":353,"threads_started":1}
I20260807 18:24:27.224329 15427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:27.311094 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15244
I20260807 18:24:27.344835 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15447
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:24:27.362582 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:27.345868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.29.156.194:35975)
0807 18:24:27.346375 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:27.346382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:27.346416 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:24:27.346477 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:27.346483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:27.346701 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:27.346882 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:27.348326 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.349420 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.350826 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.351391 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.351583 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:27.356354 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:27.356366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:27.356382 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:27.356425 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:27.359266 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.360121 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.360180 (+ 59us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.360186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.360377 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.360794 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.360799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.360803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.361196 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:27.361527 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:27.361899 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 18:24:27.362211 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260807 18:24:27.364090 15645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:27.449297 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15447
I20260807 18:24:27.471899 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15669
Aug 07 18:24:27 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:24:27.489889 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:27.473049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.29.156.195:38973)
0807 18:24:27.473733 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:27.473740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:27.473771 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:24:27.473833 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:27.473839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:27.474123 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:27.474312 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:27.475901 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.476925 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.478267 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:27.478844 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:27.478992 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:27.483771 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:27.483784 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:27.483799 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:27.483839 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:27.486768 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.487470 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.487478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.487480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.487853 (+ 373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:27.488247 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:27.488253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:27.488258 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:27.488610 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:27.488963 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:27.489244 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:24:27.489518 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":322,"threads_started":1}
I20260807 18:24:27.491185 15849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:27.576996 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15669
I20260807 18:24:27.598769 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15075
I20260807 18:24:27.600221 15166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:27.711277 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15075
I20260807 18:24:27.744594 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37073
--webserver_interface=127.29.156.254
--webserver_port=45111
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37073
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:28.086795 15873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:28.087086 15873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:28.087250 15873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:28.094681 15873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:28.094797 15873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:28.094914 15873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:28.095036 15873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:28.095121 15873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:28.105544 15873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37073
--ranger_config_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=45111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:28.106886 15873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:28.108335 15873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:28.120707 15884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:28.120757 15882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:28.120803 15886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:28.121631 15873 server_base.cc:1034] running on GCE node
I20260807 18:24:28.122493 15873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:28.124186 15873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:28.125427 15873 hybrid_clock.cc:648] HybridClock initialized: now 1786127068125391 us; error 52 us; skew 500 ppm
I20260807 18:24:28.125790 15873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:28 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127068, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:28.129212 15873 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:28.130716 15873 webserver.cc:460] Webserver started at http://127.29.156.254:45111/ using document root <none> and password file <none>
I20260807 18:24:28.131472 15873 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:28.131665 15873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:24:28 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127068, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:28.215206 15873 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:24:28.219280 15898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:28.221297 15873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:24:28.221519 15873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "3e335caee34040939571abb57b01f419"
format_stamp: "Formatted at 2026-08-07 18:24:23 on dist-test-slave-sb2z"
server_key: "ea2059f6404ae580f71f3e9ddf1eb3c7"
server_key_iv: "30835637911daf34f8320eba31557e79"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:28.222890 15873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:28.248273 15873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:28.252450 15873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:28.252722 15873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:28.272935 15873 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37073
I20260807 18:24:28.272954 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37073 every 8 connection(s)
I20260807 18:24:28.274555 15873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:28.276208 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15873
I20260807 18:24:28.277328 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:41013
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:37073
--webserver_port=34829
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:28.284840 15967 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:24:28.290144 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Bootstrap starting.
I20260807 18:24:28.310484 15967 log.cc:824] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:28.320115 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:24:28.320679 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Bootstrap complete.
I20260807 18:24:28.330392 15967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:28.331267 15967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e335caee34040939571abb57b01f419, State: Initialized, Role: FOLLOWER
I20260807 18:24:28.332046 15967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:28.332283 15967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:28.332401 15967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:28.332628 15967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:24:28.336496 15967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:28.336994 15967 leader_election.cc:302] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e335caee34040939571abb57b01f419; no voters:
I20260807 18:24:28.337666 15967 leader_election.cc:288] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:24:28.337811 15972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:24:28.339036 15972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [term 2 LEADER]: Becoming Leader. State: Replica: 3e335caee34040939571abb57b01f419, State: Running, Role: LEADER
I20260807 18:24:28.339705 15972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } }
I20260807 18:24:28.339990 15967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:28.342350 15980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e335caee34040939571abb57b01f419. Latest consensus state: current_term: 2 leader_uuid: "3e335caee34040939571abb57b01f419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } } }
I20260807 18:24:28.342762 15980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:28.343276 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3e335caee34040939571abb57b01f419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e335caee34040939571abb57b01f419" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37073 } } }
I20260807 18:24:28.343608 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:28.347213 15983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:28.354444 15983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:28.356386 15983 catalog_manager.cc:1259] Loaded cluster ID: d87bbd7bb9404484aef6853917b8521e
I20260807 18:24:28.356592 15983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:28.361867 15983 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:28.363894 15983 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Loaded TSK: 0
I20260807 18:24:28.366297 15983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:24:28.369002 15967 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 18:24:28.747592 15970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:28.748102 15970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:28.748379 15970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:28.760263 15970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:28.760500 15970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:28.760766 15970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:24:28.774186 15970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:41013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=34829
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:28.776099 15970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:28.777644 15970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:28.804577 16024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:28.810715 15970 server_base.cc:1034] running on GCE node
W20260807 18:24:28.811240 16025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:28.811878 16028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:28.812841 15970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:28.820087 15970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:28.821405 15970 hybrid_clock.cc:648] HybridClock initialized: now 1786127068821317 us; error 81 us; skew 500 ppm
I20260807 18:24:28.821902 15970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:28 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127068, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:28.831280 15970 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:24:28.847808 15970 webserver.cc:460] Webserver started at http://127.29.156.193:34829/ using document root <none> and password file <none>
I20260807 18:24:28.849413 15970 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:28.849725 15970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:24:28 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127068, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:28.895448 15970 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:24:28.913647 16044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:28.915930 15970 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.011s sys 0.007s
I20260807 18:24:28.916265 15970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "2c04216c7dc746de8d68405e561da07d"
format_stamp: "Formatted at 2026-08-07 18:24:25 on dist-test-slave-sb2z"
server_key: "5f7efd40b0aad8dc43faf6773c0799ef"
server_key_iv: "6e20b4a13a538973dc1df62b21ed09a0"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:28.917909 15970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:28.969180 15970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:28.975515 15970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:28.976013 15970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:28.977416 15970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:28.980453 15970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:28.980609 15970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:28.980756 15970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:28.980865 15970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:29.045434 15970 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:41013
I20260807 18:24:29.045555 16215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:41013 every 8 connection(s)
I20260807 18:24:29.047079 15970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:24:29.055687 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 15970
I20260807 18:24:29.056564 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:35975
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:37073
--webserver_port=43777
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:29 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127068, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:29.092986 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:29.050775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.193:41819)
0807 18:24:29.051733 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:29.051794 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:29.055018 (+ 3224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:29.067381 (+ 12363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:29.067395 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:29.068114 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:29.068959 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:29.068992 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.071990 (+ 2998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.071996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:29.074292 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:29.074312 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.076309 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.076316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:29.076625 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:29.077825 (+ 1200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:29.077861 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:29.080690 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:29.086982 (+ 6292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:29.086993 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:29.087011 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:29.087717 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:29.088574 (+ 857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:29.088580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:29.088584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:29.088863 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:29.090053 (+ 1190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:29.090061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:29.090504 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 18:24:29.090820 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 18:24:29.091433 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":1584,"threads_started":1}
I20260807 18:24:29.096050 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:29.052109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:41819 (local address 127.29.156.254:37073)
0807 18:24:29.065203 (+ 13094us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:29.065228 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:29.065298 (+ 70us) server_negotiation.cc:407] Connection header received
0807 18:24:29.065911 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:29.065942 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:29.066585 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:29.067151 (+ 566us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:29.069262 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.071699 (+ 2437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.074507 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.075657 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.075867 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:29.081113 (+ 5246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:29.081125 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:29.081150 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:29.081211 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:29.086757 (+ 5546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:29.087905 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:29.087916 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:29.087925 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:29.088416 (+ 491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:29.089010 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:29.089016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:29.089019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:29.089747 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:29.089945 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:29.093728 (+ 3783us) server_negotiation.cc:299] Negotiation successful
0807 18:24:29.094181 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12690,"thread_start_us":15446,"threads_started":1}
I20260807 18:24:29.100106 16216 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:29.100459 16216 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:29.101392 16216 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
W20260807 18:24:29.104588 16216 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:37073 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:24:29.104806 16216 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:29.105077 16216 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:29.106346 16216 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:29.106662 16216 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
W20260807 18:24:29.107751 16216 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:24:29.514113 16222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:29.514495 16222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:29.514668 16222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:29.522926 16222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:29.523151 16222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:29.523380 16222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:24:29.539227 16222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:35975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=43777
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:29.541267 16222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:29.543051 16222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:29.573077 16241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:29.574441 16240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:29.575850 16222 server_base.cc:1034] running on GCE node
W20260807 18:24:29.579856 16246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:29.580796 16222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:29.583237 16222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:29.584522 16222 hybrid_clock.cc:648] HybridClock initialized: now 1786127069584445 us; error 71 us; skew 500 ppm
I20260807 18:24:29.584897 16222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:29 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:29.588408 16222 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:24:29.593009 16222 webserver.cc:460] Webserver started at http://127.29.156.194:43777/ using document root <none> and password file <none>
I20260807 18:24:29.593782 16222 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:29.593961 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:24:29 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:29.615819 16222 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:24:29.618970 16259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:29.620814 16222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:24:29.621042 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "79f0c48de41c43ff8f274cb93b0ef99d"
format_stamp: "Formatted at 2026-08-07 18:24:26 on dist-test-slave-sb2z"
server_key: "d3c776dd53ffea4a71c665989327aaed"
server_key_iv: "4ffcdace3b848cd3e7aeae8a8e1fed04"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:29.622112 16222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 18:24:29 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:29.643632 16222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:29.648116 16222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:29.648569 16222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:29.649873 16222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:29.658051 16222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:29.658233 16222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:24:29.658387 16222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:29.658510 16222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:29.733361 16222 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:35975
I20260807 18:24:29.733419 16410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:35975 every 8 connection(s)
I20260807 18:24:29.734982 16222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:24:29.736979 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 16222
I20260807 18:24:29.738022 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.156.212:50475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:38973
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:37073
--webserver_port=39607
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:29 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:29.788564 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:29.750803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:59813 (local address 127.29.156.254:37073)
0807 18:24:29.766747 (+ 15944us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:29.766752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:29.766769 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:24:29.766823 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:29.766827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:29.766892 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:29.767039 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:29.769424 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.770760 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.773517 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.774310 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.774431 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:29.779415 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:29.779425 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:29.779432 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:29.779496 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:29.783672 (+ 4176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:29.784883 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:29.784892 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:29.784899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:29.785091 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:29.785799 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:29.785808 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:29.785813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:29.786286 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:29.786469 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:29.787728 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0807 18:24:29.788018 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15817,"thread_start_us":15692,"threads_started":1}
I20260807 18:24:29.791720 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:29.750154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.194:59813)
0807 18:24:29.762192 (+ 12038us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:29.762267 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:29.765826 (+ 3559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:29.767389 (+ 1563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:29.767406 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:29.768203 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:29.769163 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:29.769200 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.770979 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.770987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:29.773313 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:29.773332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:29.774863 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:29.774872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:29.775216 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:29.776071 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:29.776106 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:29.779115 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:29.783883 (+ 4768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:29.783896 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:29.783920 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:29.784684 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:29.785274 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:29.785282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:29.785289 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:29.785643 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:29.786584 (+ 941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:29.786596 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:29.787159 (+ 563us) client_negotiation.cc:769] Sending connection context
0807 18:24:29.789220 (+ 2061us) client_negotiation.cc:240] Negotiation successful
0807 18:24:29.789956 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11558,"thread_start_us":12154,"threads_started":1}
I20260807 18:24:29.794626 16411 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:29.795008 16411 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:29.796155 16411 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:29.799186 15919 ts_manager.cc:194] Registered new tserver with Master: 79f0c48de41c43ff8f274cb93b0ef99d (127.29.156.194:35975)
I20260807 18:24:29.801501 15919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:59813
I20260807 18:24:30.107930 16216 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:30.108487 16216 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:30.110174 15919 ts_manager.cc:194] Registered new tserver with Master: 2c04216c7dc746de8d68405e561da07d (127.29.156.193:41013)
I20260807 18:24:30.111732 15919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:41819
Aug 07 18:24:30 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
W20260807 18:24:30.255919 16415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:30.256459 16415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:30.256693 16415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:30.267935 16415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:30.268165 16415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:30.268401 16415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:24:30.284885 16415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.156.212:50475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:38973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=39607
--tserver_master_addrs=127.29.156.254:37073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:30.286931 16415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:30.288748 16415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:30.342379 16431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:30.342944 16415 server_base.cc:1034] running on GCE node
W20260807 18:24:30.343909 16420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:30.343912 16421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:30.344875 16415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:30.355844 16415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:30.363768 16415 hybrid_clock.cc:648] HybridClock initialized: now 1786127070363623 us; error 120 us; skew 500 ppm
I20260807 18:24:30.364375 16415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:30 dist-test-slave-sb2z krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127070, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:30.369185 16415 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:24:30.386862 16415 webserver.cc:460] Webserver started at http://127.29.156.195:39607/ using document root <none> and password file <none>
I20260807 18:24:30.388029 16415 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:30.388273 16415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:24:30 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127070, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:30.433674 16415 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.005s sys 0.009s
I20260807 18:24:30.444187 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:30.446202 16415 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.003s
I20260807 18:24:30.446468 16415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "173d1fb964ad4703b0881119dace43e7"
format_stamp: "Formatted at 2026-08-07 18:24:27 on dist-test-slave-sb2z"
server_key: "86bf5771943ac7278e35cad070e951f2"
server_key_iv: "30eb04cb1f1288ff8354ad47c0fa52ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:24:30.448012 16415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:30.490469 16415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:30.495813 16415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:30.496220 16415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:30.503674 16415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:30.513124 16415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:30.513360 16415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:30.513546 16415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:30.513676 16415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:24:30 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127069, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for HTTP/127.29.156.212@KRBTEST.COM
I20260807 18:24:30.783170 16415 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:38973
I20260807 18:24:30.783218 16614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:38973 every 8 connection(s)
I20260807 18:24:30.784852 16415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:24:30.792869 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 16415
Aug 07 18:24:30 dist-test-slave-sb2z krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127070, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:30.806751 16411 heartbeater.cc:503] Master 127.29.156.254:37073 was elected leader, sending a full tablet report...
I20260807 18:24:30.810487 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.787781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43487 (local address 127.29.156.254:37073)
0807 18:24:30.788310 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.788318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.789403 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 18:24:30.791246 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.791253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.791358 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.791499 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:30.793353 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.794897 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.798145 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.798655 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.798809 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.803025 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:30.803032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:30.803037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:30.803096 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:30.806401 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.807452 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.807458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.807461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.807652 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.808253 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.808258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.808261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.808697 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:30.809143 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.809842 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.810087 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":264,"threads_started":1}
I20260807 18:24:30.811625 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.788356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37073 (local address 127.29.156.195:43487)
0807 18:24:30.789199 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.789254 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.790954 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.791888 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.791898 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.792425 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:30.793149 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.793173 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.795326 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.795336 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.797963 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.797977 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.798925 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.798937 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.799213 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.800151 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:30.800195 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:30.802774 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:30.806590 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:30.806601 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:30.806619 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:30.807314 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:30.807822 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:30.807830 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:30.807836 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:30.808113 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:30.809097 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:30.809107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:30.809464 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.809746 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.810322 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":359,"threads_started":1}
I20260807 18:24:30.813417 16615 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37073
I20260807 18:24:30.813753 16615 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:30.814658 16615 heartbeater.cc:511] Master 127.29.156.254:37073 requested a full tablet report, sending...
I20260807 18:24:30.814992 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.794109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.29.156.254:37073)
0807 18:24:30.797531 (+ 3422us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.797537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.797867 (+ 330us) server_negotiation.cc:407] Connection header received
0807 18:24:30.798101 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.798106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.798201 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.798325 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:30.800189 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.801027 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.802844 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.803727 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.803869 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.806929 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:30.806936 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:30.806940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:30.806990 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:30.810025 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.812824 (+ 2799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.812829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.812832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.813009 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.813458 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.813462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.813464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.813771 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:30.814014 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.814494 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.814663 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3258,"thread_start_us":3078,"threads_started":1}
I20260807 18:24:30.816359 15919 ts_manager.cc:194] Registered new tserver with Master: 173d1fb964ad4703b0881119dace43e7 (127.29.156.195:38973)
I20260807 18:24:30.817584 15919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:43487
I20260807 18:24:30.818039 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:24:30.834863 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.821243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.29.156.254:37073)
0807 18:24:30.821445 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.821453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.821881 (+ 428us) server_negotiation.cc:407] Connection header received
0807 18:24:30.822061 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.822065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.822120 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.822198 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:30.824002 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.824747 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.826078 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.826471 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.826567 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.829293 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:30.829298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:30.829301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:30.829343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:30.832166 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.832831 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.832835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.832837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.833059 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:30.833460 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:30.833464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:30.833466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:30.833730 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:30.833922 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.834326 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.834513 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:24:30.840623 15919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:24:30.848673 15919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:24:30.888798 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.863793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.29.156.194:35975)
0807 18:24:30.880252 (+ 16459us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.880261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.880295 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:24:30.880405 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.880413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.880736 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.881019 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:30.881813 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.883598 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.886065 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.887220 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.887437 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.887654 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.888269 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.888444 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16237,"thread_start_us":17304,"threads_started":1}
I20260807 18:24:30.888909 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.863640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:35975 (local address 127.0.0.1:46138)
0807 18:24:30.873199 (+ 9559us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.873248 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.873435 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.881039 (+ 7604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.881048 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.881099 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:30.881568 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.881584 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.883726 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.883731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.885865 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.885880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.887823 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.887829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.888034 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.888073 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.888223 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.888437 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9280,"mutex_wait_us":33,"thread_start_us":18130,"threads_started":1}
I20260807 18:24:30.889711 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.863738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38973 (local address 127.0.0.1:46850)
0807 18:24:30.878474 (+ 14736us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.878508 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.878659 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.881824 (+ 3165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.881829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.881854 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:30.882231 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.882246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.884377 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.884382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.886814 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.886830 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.888867 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.888876 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.889052 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.889085 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.889158 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.889309 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14563,"mutex_wait_us":1,"thread_start_us":9045,"threads_started":1}
I20260807 18:24:30.890225 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.864429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.29.156.195:38973)
0807 18:24:30.881222 (+ 16793us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.881233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.881267 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:24:30.881365 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.881372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.881611 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.882444 (+ 833us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:30.882943 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.884227 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.887039 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.888584 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.889485 (+ 901us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.889656 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.889737 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.889893 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16616,"thread_start_us":16450,"threads_started":1}
I20260807 18:24:30.890324 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.866291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.29.156.193:41013)
0807 18:24:30.880655 (+ 14364us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.880667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.881733 (+ 1066us) server_negotiation.cc:407] Connection header received
0807 18:24:30.882004 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.882010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.882164 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.882351 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:30.882983 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.884593 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.887245 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.888719 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.888901 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.889069 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.889818 (+ 749us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.889994 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14120,"thread_start_us":13897,"threads_started":1}
I20260807 18:24:30.890628 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.865984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:41013 (local address 127.0.0.1:35888)
0807 18:24:30.881609 (+ 15625us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.881626 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.881772 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.882341 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.882346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.882372 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:30.882781 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.882798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.884750 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.884755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.887065 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.887077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.889237 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.889243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.889403 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.889442 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.889539 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.889678 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15520,"thread_start_us":12770,"threads_started":1}
I20260807 18:24:30.893509 16327 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (DEFAULT_TABLE table=test-table [id=40f8aea477064b19951bbc8f4d7817da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:30.894052 16516 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (DEFAULT_TABLE table=test-table [id=40f8aea477064b19951bbc8f4d7817da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:30.894872 16327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:30.894955 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (DEFAULT_TABLE table=test-table [id=40f8aea477064b19951bbc8f4d7817da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:30.895517 16516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:30.896330 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:30.907719 16641 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Bootstrap starting.
I20260807 18:24:30.911250 16641 tablet_bootstrap.cc:654] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:30.911610 16645 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Bootstrap starting.
I20260807 18:24:30.912510 16646 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Bootstrap starting.
I20260807 18:24:30.913331 16641 log.cc:824] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:30.915869 16645 tablet_bootstrap.cc:654] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:30.916369 16646 tablet_bootstrap.cc:654] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:30.916961 16645 log.cc:824] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:30.917457 16641 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: No bootstrap required, opened a new log
I20260807 18:24:30.917810 16641 ts_tablet_manager.cc:1397] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 18:24:30.917769 16646 log.cc:824] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:30.920252 16645 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: No bootstrap required, opened a new log
I20260807 18:24:30.920557 16645 ts_tablet_manager.cc:1397] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:24:30.922463 16646 tablet_bootstrap.cc:492] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: No bootstrap required, opened a new log
I20260807 18:24:30.922747 16646 ts_tablet_manager.cc:1397] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:24:30.929396 16645 raft_consensus.cc:348] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.929811 16645 raft_consensus.cc:374] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:30.929519 16641 raft_consensus.cc:348] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.929953 16641 raft_consensus.cc:374] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:30.929951 16645 raft_consensus.cc:729] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c04216c7dc746de8d68405e561da07d, State: Initialized, Role: FOLLOWER
I20260807 18:24:30.930071 16641 raft_consensus.cc:729] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f0c48de41c43ff8f274cb93b0ef99d, State: Initialized, Role: FOLLOWER
I20260807 18:24:30.930665 16645 consensus_queue.cc:260] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.930774 16641 consensus_queue.cc:260] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.931849 16216 heartbeater.cc:503] Master 127.29.156.254:37073 was elected leader, sending a full tablet report...
I20260807 18:24:30.933816 16645 ts_tablet_manager.cc:1428] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:24:30.934818 16646 raft_consensus.cc:348] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.935379 16646 raft_consensus.cc:374] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:30.935622 16646 raft_consensus.cc:729] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173d1fb964ad4703b0881119dace43e7, State: Initialized, Role: FOLLOWER
I20260807 18:24:30.936338 16646 consensus_queue.cc:260] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.936882 16641 ts_tablet_manager.cc:1428] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Time spent starting tablet: real 0.019s user 0.016s sys 0.001s
I20260807 18:24:30.937706 16615 heartbeater.cc:503] Master 127.29.156.254:37073 was elected leader, sending a full tablet report...
I20260807 18:24:30.938764 16646 ts_tablet_manager.cc:1428] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260807 18:24:30.949052 16656 raft_consensus.cc:482] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:24:30.949422 16656 raft_consensus.cc:504] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.951097 16656 leader_election.cc:288] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79f0c48de41c43ff8f274cb93b0ef99d (127.29.156.194:35975), 173d1fb964ad4703b0881119dace43e7 (127.29.156.195:38973)
I20260807 18:24:30.960708 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.951661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:40219 (local address 127.29.156.195:38973)
0807 18:24:30.951896 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.951903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.952664 (+ 761us) server_negotiation.cc:407] Connection header received
0807 18:24:30.952914 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.952918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.952983 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.953100 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:30.953743 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.954936 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.958231 (+ 3295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.959555 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.959737 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.959811 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.960315 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.960448 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:24:30.960709 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.951774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:42447 (local address 127.29.156.194:35975)
0807 18:24:30.951991 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:30.951997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:30.953061 (+ 1064us) server_negotiation.cc:407] Connection header received
0807 18:24:30.953232 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:30.953240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:30.953318 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:30.953390 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:30.954110 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.955240 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.958231 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.959611 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.959754 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.959817 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:30.960315 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 18:24:30.960448 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:24:30.961149 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.951649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38973 (local address 127.29.156.193:40219)
0807 18:24:30.952516 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.952539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.952745 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.953132 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.953139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.953167 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:30.953597 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.953610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.955126 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.955132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.958042 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.958062 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.959842 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.959856 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.960091 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.960163 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.960277 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.960505 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":96,"thread_start_us":493,"threads_started":1}
I20260807 18:24:30.961313 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:30.952155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:35975 (local address 127.29.156.193:42447)
0807 18:24:30.952958 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:30.952971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:30.953099 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:30.953444 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:30.953461 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:30.953504 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:30.953872 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.953881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.955405 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.955411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.958052 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:30.958062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:30.959870 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:30.959875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:30.960147 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:30.960175 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:24:30.960277 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:24:30.960505 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":157,"thread_start_us":475,"threads_started":1}
I20260807 18:24:30.962172 16540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdbb9e5b8b24bf4a360a17f71ff56e7" candidate_uuid: "2c04216c7dc746de8d68405e561da07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173d1fb964ad4703b0881119dace43e7" is_pre_election: true
I20260807 18:24:30.962316 16352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdbb9e5b8b24bf4a360a17f71ff56e7" candidate_uuid: "2c04216c7dc746de8d68405e561da07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" is_pre_election: true
I20260807 18:24:30.962707 16540 raft_consensus.cc:2454] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c04216c7dc746de8d68405e561da07d in term 0.
I20260807 18:24:30.962942 16352 raft_consensus.cc:2454] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c04216c7dc746de8d68405e561da07d in term 0.
I20260807 18:24:30.963361 16048 leader_election.cc:302] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173d1fb964ad4703b0881119dace43e7, 2c04216c7dc746de8d68405e561da07d; no voters:
I20260807 18:24:30.963836 16656 raft_consensus.cc:2781] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:24:30.964011 16656 raft_consensus.cc:482] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:24:30.964162 16656 raft_consensus.cc:3037] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:30.966632 16656 raft_consensus.cc:504] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.967336 16656 leader_election.cc:288] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [CANDIDATE]: Term 1 election: Requested vote from peers 79f0c48de41c43ff8f274cb93b0ef99d (127.29.156.194:35975), 173d1fb964ad4703b0881119dace43e7 (127.29.156.195:38973)
I20260807 18:24:30.967808 16352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdbb9e5b8b24bf4a360a17f71ff56e7" candidate_uuid: "2c04216c7dc746de8d68405e561da07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f0c48de41c43ff8f274cb93b0ef99d"
I20260807 18:24:30.967833 16540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdbb9e5b8b24bf4a360a17f71ff56e7" candidate_uuid: "2c04216c7dc746de8d68405e561da07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173d1fb964ad4703b0881119dace43e7"
I20260807 18:24:30.968047 16352 raft_consensus.cc:3037] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:30.968009 16540 raft_consensus.cc:3037] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:30.970304 16540 raft_consensus.cc:2454] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c04216c7dc746de8d68405e561da07d in term 1.
I20260807 18:24:30.970450 16352 raft_consensus.cc:2454] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c04216c7dc746de8d68405e561da07d in term 1.
I20260807 18:24:30.970712 16048 leader_election.cc:302] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173d1fb964ad4703b0881119dace43e7, 2c04216c7dc746de8d68405e561da07d; no voters:
I20260807 18:24:30.970992 16656 raft_consensus.cc:2781] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:30.971598 16656 raft_consensus.cc:686] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 1 LEADER]: Becoming Leader. State: Replica: 2c04216c7dc746de8d68405e561da07d, State: Running, Role: LEADER
I20260807 18:24:30.972080 16656 consensus_queue.cc:237] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } }
I20260807 18:24:30.976660 15917 catalog_manager.cc:5495] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c04216c7dc746de8d68405e561da07d (127.29.156.193). New cstate: current_term: 1 leader_uuid: "2c04216c7dc746de8d68405e561da07d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c04216c7dc746de8d68405e561da07d" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 41013 } health_report { overall_health: HEALTHY } } }
W20260807 18:24:30.995896 16412 tablet.cc:2369] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:31.014242 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:31.008535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.29.156.193:41013)
0807 18:24:31.008784 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:31.008789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:31.008808 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:24:31.008983 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:31.008987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:31.009069 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:31.009202 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:24:31.009664 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.010478 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.011939 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.012440 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.012549 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:31.013097 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:24:31.013636 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:24:31.013761 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:31.013886 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:24:31.013960 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:24:31.024191 16352 raft_consensus.cc:1261] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 1 FOLLOWER]: Refusing update from remote peer 2c04216c7dc746de8d68405e561da07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:31.024259 16540 raft_consensus.cc:1261] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 1 FOLLOWER]: Refusing update from remote peer 2c04216c7dc746de8d68405e561da07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:31.025313 16656 consensus_queue.cc:1035] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "173d1fb964ad4703b0881119dace43e7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:31.025671 16661 consensus_queue.cc:1035] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "79f0c48de41c43ff8f274cb93b0ef99d" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:24:31.036715 16616 tablet.cc:2369] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:31.040589 16665 mvcc.cc:205] Tried to move back new op lower bound from 7315976482903527424 to 7315976482705723392. Current Snapshot: MvccSnapshot[applied={T|T < 7315976482903527424}]
I20260807 18:24:31.044698 16667 mvcc.cc:205] Tried to move back new op lower bound from 7315976482903527424 to 7315976482705723392. Current Snapshot: MvccSnapshot[applied={T|T < 7315976482903527424}]
I20260807 18:24:31.058118 15918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:31.058502 15918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:31.063570 15918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e335caee34040939571abb57b01f419: Sending DeleteTablet for 3 replicas of tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7
I20260807 18:24:31.064975 16327 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46138
I20260807 18:24:31.064975 16516 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46850
I20260807 18:24:31.065793 16119 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:31 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:35888
I20260807 18:24:31.073604 16685 tablet_replica.cc:330] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: stopping tablet replica
I20260807 18:24:31.074136 16685 raft_consensus.cc:2229] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:31.074729 16685 raft_consensus.cc:2258] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:31.075655 16683 tablet_replica.cc:330] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: stopping tablet replica
I20260807 18:24:31.076205 16683 raft_consensus.cc:2229] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:31.076838 16683 raft_consensus.cc:2258] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:31.077826 16686 tablet_replica.cc:330] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: stopping tablet replica
I20260807 18:24:31.078305 16686 raft_consensus.cc:2229] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:24:31.078763 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15970
I20260807 18:24:31.079098 16686 raft_consensus.cc:2258] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:31.081369 16685 ts_tablet_manager.cc:1905] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:31.081595 16686 ts_tablet_manager.cc:1905] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:31.089432 16685 ts_tablet_manager.cc:1918] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:31.089867 16685 log.cc:1196] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/2bdbb9e5b8b24bf4a360a17f71ff56e7
I20260807 18:24:31.090348 16686 ts_tablet_manager.cc:1918] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:31.090605 16685 ts_tablet_manager.cc:1939] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 173d1fb964ad4703b0881119dace43e7: Deleting consensus metadata
I20260807 18:24:31.090677 16686 log.cc:1196] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/2bdbb9e5b8b24bf4a360a17f71ff56e7
I20260807 18:24:31.091619 16686 ts_tablet_manager.cc:1939] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 2c04216c7dc746de8d68405e561da07d: Deleting consensus metadata
I20260807 18:24:31.093005 15899 catalog_manager.cc:4841] TS 173d1fb964ad4703b0881119dace43e7 (127.29.156.195:38973): tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (table test-table [id=40f8aea477064b19951bbc8f4d7817da]) successfully deleted
I20260807 18:24:31.093618 15901 catalog_manager.cc:4841] TS 2c04216c7dc746de8d68405e561da07d (127.29.156.193:41013): tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (table test-table [id=40f8aea477064b19951bbc8f4d7817da]) successfully deleted
I20260807 18:24:31.094269 16683 ts_tablet_manager.cc:1905] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:31.100577 16683 ts_tablet_manager.cc:1918] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:31.100857 16683 log.cc:1196] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/2bdbb9e5b8b24bf4a360a17f71ff56e7
I20260807 18:24:31.101743 16683 ts_tablet_manager.cc:1939] T 2bdbb9e5b8b24bf4a360a17f71ff56e7 P 79f0c48de41c43ff8f274cb93b0ef99d: Deleting consensus metadata
I20260807 18:24:31.101768 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:31.080995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.29.156.193:41013)
0807 18:24:31.081204 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:31.081211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:31.081235 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:24:31.081304 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:31.081309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:31.081401 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:31.081508 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:31.083921 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.085342 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.087634 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.088516 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.088631 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:31.092853 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:31.092867 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:31.092886 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:31.092936 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:31.097918 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.098945 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.098956 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.098960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.099162 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.099747 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.099753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.099757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.100271 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:31.100621 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:31.101054 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 18:24:31.101333 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:24:31.102664 16210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:31.104089 15902 catalog_manager.cc:4841] TS 79f0c48de41c43ff8f274cb93b0ef99d (127.29.156.194:35975): tablet 2bdbb9e5b8b24bf4a360a17f71ff56e7 (table test-table [id=40f8aea477064b19951bbc8f4d7817da]) successfully deleted
I20260807 18:24:31.209877 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15970
I20260807 18:24:31.237350 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 16222
I20260807 18:24:31.252833 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:31.238232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.29.156.194:35975)
0807 18:24:31.238407 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:31.238411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:31.238481 (+ 70us) server_negotiation.cc:407] Connection header received
0807 18:24:31.238642 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:31.238645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:31.238729 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:31.238817 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:31.240441 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.241329 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.242607 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.243062 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.243169 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:31.246215 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:31.246232 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:31.246250 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:31.246288 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:31.249762 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.250561 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.250569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.250571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.250713 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.251173 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.251177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.251179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.251762 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:31.252048 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:31.252332 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:24:31.252521 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:24:31.253671 16406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:31.351784 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 16222
I20260807 18:24:31.377400 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 16415
I20260807 18:24:31.392766 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:31.378308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36780 (local address 127.29.156.195:38973)
0807 18:24:31.378543 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:31.378550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:31.378602 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:24:31.378834 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:31.378844 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:31.378912 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:31.379035 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:31.380584 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.381361 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.382708 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:31.383123 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:31.383265 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:31.386283 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:31.386298 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:31.386315 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:31.386355 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:31.389618 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.390431 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.390440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.390443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.390599 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:31.390966 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:31.390970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:31.390972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:31.391390 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:31.391741 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:31.392163 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 18:24:31.392359 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:24:31.393567 16610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:31.484639 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 16415
I20260807 18:24:31.510589 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 15873
I20260807 18:24:31.512074 15962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:31.627807 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 15873
I20260807 18:24:31.662693 30323 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:24:32.009068 30323 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:24:32.009343 30323 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:24:32.365370 30323 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:24:32.365 UTC [14468] LOG: received smart shutdown request
2026-08-07 18:24:32.369 UTC [14468] LOG: background worker "logical replication launcher" (PID 14484) exited with exit code 1
2026-08-07 18:24:32.369 UTC [14475] LOG: shutting down
2026-08-07 18:24:32.557 UTC [14468] LOG: database system is shut down
2026-08-07T18:24:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (58251 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:24:32 dist-test-slave-sb2z krb5kdc[16705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:24:32 dist-test-slave-sb2z krb5kdc[16705](info): set up 2 sockets
Aug 07 18:24:32 dist-test-slave-sb2z krb5kdc[16705](info): commencing operation
krb5kdc: starting...
W20260807 18:24:34.063552 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:24:34 dist-test-slave-sb2z krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:24:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:24:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:24:34.373591 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46161
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:34.725317 16722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:34.725620 16722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:34.725792 16722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:34.733490 16722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:34.733613 16722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:34.733718 16722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:34.733822 16722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:34.733907 16722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:34.744063 16722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:34.745527 16722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:34.746999 16722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:34.759948 16728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:34.759976 16730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:34.761386 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:34.762396 16722 server_base.cc:1034] running on GCE node
I20260807 18:24:34.763478 16722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:34.765153 16722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:34.766425 16722 hybrid_clock.cc:648] HybridClock initialized: now 1786127074766379 us; error 55 us; skew 500 ppm
I20260807 18:24:34.766863 16722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:34 dist-test-slave-sb2z krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:34.775064 16722 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:34.777313 16722 webserver.cc:460] Webserver started at http://127.29.156.254:35509/ using document root <none> and password file <none>
I20260807 18:24:34.778431 16722 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:34.778596 16722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:34.779059 16722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:34.783416 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "5783003b6fed41b3b67b30c038a6f120"
format_stamp: "Formatted at 2026-08-07 18:24:34 on dist-test-slave-sb2z"
server_key: "37b6b6e17e87ff0a1578455d3859281b"
server_key_iv: "18d3e42afca42b8a77cbac17dffa6cc1"
server_key_version: "encryptionkey@0"
I20260807 18:24:34.784746 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "5783003b6fed41b3b67b30c038a6f120"
format_stamp: "Formatted at 2026-08-07 18:24:34 on dist-test-slave-sb2z"
server_key: "37b6b6e17e87ff0a1578455d3859281b"
server_key_iv: "18d3e42afca42b8a77cbac17dffa6cc1"
server_key_version: "encryptionkey@0"
I20260807 18:24:34.792649 16722 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:24:34.797286 16740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:34.799451 16722 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:24:34.799705 16722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "5783003b6fed41b3b67b30c038a6f120"
format_stamp: "Formatted at 2026-08-07 18:24:34 on dist-test-slave-sb2z"
server_key: "37b6b6e17e87ff0a1578455d3859281b"
server_key_iv: "18d3e42afca42b8a77cbac17dffa6cc1"
server_key_version: "encryptionkey@0"
I20260807 18:24:34.799997 16722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:34.825205 16722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:34.830435 16722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:34.830775 16722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:34.852319 16722 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:46161
I20260807 18:24:34.852349 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:46161 every 8 connection(s)
I20260807 18:24:34.853961 16722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:34.855687 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 16722
I20260807 18:24:34.855939 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:24:34.856801 30323 external_mini_cluster.cc:1428] Setting key 1d9c9ccb54add5203f526f7712730231
I20260807 18:24:34.858907 16816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:34.869202 16816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:34 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.156.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.253@KRBTEST.COM" created.
I20260807 18:24:34.894807 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.872412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46161 (local address 127.0.0.1:50326)
0807 18:24:34.873296 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:34.873343 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:34.875516 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:34.876617 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:34.876622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:34.877191 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:34.877545 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:34.877562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:34.880222 (+ 2660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:34.880226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:34.881330 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:34.881341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:34.882877 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:34.882882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:34.883015 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:34.883913 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:34.883944 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:34.886204 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:34.890464 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:34.890474 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:34.890484 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:34.890905 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:34.891517 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:34.891542 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:34.891548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:34.891781 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:34.892869 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:34.892875 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:34.893071 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 18:24:34.893296 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 18:24:34.893702 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":387,"threads_started":1}
I20260807 18:24:34.894835 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.873763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.29.156.254:46161)
0807 18:24:34.874635 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:34.874647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:34.874718 (+ 71us) server_negotiation.cc:407] Connection header received
0807 18:24:34.875966 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:34.875978 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:34.876334 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:34.876600 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:34.877757 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:34.880048 (+ 2291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:34.881521 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:34.882622 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:34.882854 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:34.886486 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:34.886501 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:34.886523 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:34.886573 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:34.890332 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:34.891055 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:34.891061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:34.891064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:34.891337 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:34.891934 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:34.891939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:34.891942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:34.892724 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:34.892945 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:34.893513 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 18:24:34.893882 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":601,"spinlock_wait_cycles":32512,"thread_start_us":405,"threads_started":1}
I20260807 18:24:34.898406 16816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:24:34.899659 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.899066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35564)
0807 18:24:34.899365 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:34.899478 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
W20260807 18:24:34.899935 16743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
I20260807 18:24:34.901579 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.901213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35572)
0807 18:24:34.901375 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:34.901453 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:24:34.901777 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:40061: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
I20260807 18:24:34.927824 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:40061
--webserver_interface=127.29.156.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:34.951112 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } attempt: 1
I20260807 18:24:34.952440 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.951960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35586)
0807 18:24:34.952177 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:34.952297 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:24:34.953552 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:34.953157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35602)
0807 18:24:34.953321 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:34.953406 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:24:34.953830 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:40061: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
I20260807 18:24:35.022107 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } attempt: 2
I20260807 18:24:35.023411 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.022903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35608)
0807 18:24:35.023177 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.023268 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:24:35.024573 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.024190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35618)
0807 18:24:35.024347 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.024433 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:24:35.024847 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:40061: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
I20260807 18:24:35.116166 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } attempt: 3
I20260807 18:24:35.117460 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.116994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35628)
0807 18:24:35.117224 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.117321 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:24:35.118589 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.118218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35644)
0807 18:24:35.118370 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.118457 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:24:35.118884 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:40061: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
I20260807 18:24:35.262259 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } attempt: 4
I20260807 18:24:35.263662 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.263006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35658)
0807 18:24:35.263341 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.263437 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 18:24:35.264825 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.264432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35674)
0807 18:24:35.264598 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.264703 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:24:35.265060 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:40061: Network error: Client connection negotiation failed: client connection to 127.29.156.253:40061: connect: Connection refused (error 111)
W20260807 18:24:35.278137 16823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:35.278502 16823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:35.278679 16823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:35.286504 16823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:35.286679 16823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:35.286795 16823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:35.286907 16823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:35.286998 16823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:35.297834 16823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:40061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.156.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:35.299342 16823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:35.300827 16823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:35.313719 16828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:35.313763 16833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:35.313815 16830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:35.314889 16823 server_base.cc:1034] running on GCE node
I20260807 18:24:35.315799 16823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:35.317727 16823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:35.318967 16823 hybrid_clock.cc:648] HybridClock initialized: now 1786127075318901 us; error 61 us; skew 500 ppm
I20260807 18:24:35.319353 16823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:35.324134 16823 init.cc:377] Logged in from keytab as kudu/127.29.156.253@KRBTEST.COM (short username kudu)
I20260807 18:24:35.326421 16823 webserver.cc:460] Webserver started at http://127.29.156.253:35007/ using document root <none> and password file <none>
I20260807 18:24:35.327203 16823 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:35.327342 16823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:35.327750 16823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:35.330821 16823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data/instance:
uuid: "35a111225768484ca19f10751f3de66c"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "1d53d864e6f97f8f213b3feeacee235f"
server_key_iv: "20d16d733c2b98cb6490415ff67ae789"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.331719 16823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal/instance:
uuid: "35a111225768484ca19f10751f3de66c"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "1d53d864e6f97f8f213b3feeacee235f"
server_key_iv: "20d16d733c2b98cb6490415ff67ae789"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.337250 16823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:24:35.340806 16848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:35.342190 16823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:24:35.342389 16823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "35a111225768484ca19f10751f3de66c"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "1d53d864e6f97f8f213b3feeacee235f"
server_key_iv: "20d16d733c2b98cb6490415ff67ae789"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.342602 16823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:35.365456 16823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:35.369947 16823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:35.370265 16823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:35.390447 16823 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.253:40061
I20260807 18:24:35.390542 16926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.253:40061 every 8 connection(s)
I20260807 18:24:35.391939 16823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
I20260807 18:24:35.395792 16927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:35.397899 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 16823
I20260807 18:24:35.398123 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-1/wal/instance
I20260807 18:24:35.398710 30323 external_mini_cluster.cc:1428] Setting key 3779f24eccd355a50b1115c486c40975
I20260807 18:24:35.406211 16927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.156.252@KRBTEST.COM; defaulting to no policy
I20260807 18:24:35.430430 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.409994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.29.156.254:46161)
0807 18:24:35.410633 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.410643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.411610 (+ 967us) server_negotiation.cc:407] Connection header received
0807 18:24:35.413636 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.413642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.413753 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.413881 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.415486 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.416358 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.417966 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.418403 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.418551 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.422910 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:35.422916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:35.422919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:35.422960 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:35.426319 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.427620 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.427625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.427627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.427757 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.428350 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.428353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.428355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.428677 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:35.428793 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:35.429918 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0807 18:24:35.430099 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":315,"threads_started":1}
I20260807 18:24:35.431941 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.410584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46161 (local address 127.0.0.1:50334)
0807 18:24:35.411417 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.411470 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:35.413330 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:35.414148 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:35.414157 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:35.414710 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:35.415298 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.415316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.416544 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.416548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.417805 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.417819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.418667 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.418675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.418906 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.419779 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:35.419804 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:35.422609 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:35.426529 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.426543 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.426567 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.427439 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.427896 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.427904 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.427909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.428200 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.428834 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:35.428844 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:35.429397 (+ 553us) client_negotiation.cc:769] Sending connection context
0807 18:24:35.429810 (+ 413us) client_negotiation.cc:240] Negotiation successful
0807 18:24:35.430434 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":362,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.252@KRBTEST.COM" created.
I20260807 18:24:35.434218 16927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:24:35.452026 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.435339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.29.156.253:40061)
0807 18:24:35.436106 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.436112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.436139 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:24:35.436185 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.436191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.436377 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.436532 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.437581 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.438691 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.440677 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.441296 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.441413 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.445580 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:35.445597 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:35.445615 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:35.445653 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:35.448791 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.449650 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.449657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.449660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.449841 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.450282 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.450286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.450288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.450660 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:35.450832 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:35.451322 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:24:35.451711 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":601,"thread_start_us":438,"threads_started":1}
I20260807 18:24:35.452165 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.435009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35688)
0807 18:24:35.435305 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.435327 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:35.435454 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:35.436595 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:35.436602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:35.437141 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:35.437367 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.437382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.438854 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.438863 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.440465 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.440485 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.441475 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.441488 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.441763 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.442506 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:35.442536 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:35.445318 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:35.448942 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.448950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.448957 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.449509 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.449957 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.449964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.449971 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.450170 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.450795 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:35.450803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:35.450996 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 18:24:35.451171 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:24:35.451584 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 18:24:35.454664 16927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:24:35.455966 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.455336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55444)
0807 18:24:35.455575 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.455746 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 18:24:35.456285 16849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.458247 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.457713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55454)
0807 18:24:35.457956 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.458091 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:24:35.458513 16927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
I20260807 18:24:35.475921 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:36123
--webserver_interface=127.29.156.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:35.507886 16927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 1
I20260807 18:24:35.509207 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.508701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55462)
0807 18:24:35.508919 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.509036 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:24:35.510281 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.509898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55478)
0807 18:24:35.510045 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.510163 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 18:24:35.510511 16927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.564352 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } attempt: 5
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:24:35.578745 16927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 2
I20260807 18:24:35.579905 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.579373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55490)
0807 18:24:35.579611 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.579708 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:24:35.580348 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.565191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35696)
0807 18:24:35.565475 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.565501 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:35.565643 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:35.566100 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:35.566105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:35.566698 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:35.566933 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.566948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.568015 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.568021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.569367 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.569383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.570092 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.570098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.570285 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.570998 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:35.571021 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:35.573522 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:35.577612 (+ 4090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.577616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.577619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.578003 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.578407 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.578410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.578412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.578543 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.579203 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:35.579210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:35.579389 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 18:24:35.579662 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 18:24:35.580003 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:24:35.580763 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.565377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.29.156.253:40061)
0807 18:24:35.565606 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.565611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.565633 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:24:35.565837 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.565841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.565913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.566008 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.567143 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.567875 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.569515 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.569932 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.570026 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.573735 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:35.573742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:35.573747 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:35.573788 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:35.577411 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.578133 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.578139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.578143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.578297 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.578653 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.578658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.578661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.579062 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:35.579216 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:35.579785 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:24:35.580189 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:24:35.581077 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.580662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55498)
0807 18:24:35.580827 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.580939 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 18:24:35.581328 16927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.581691 16816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:24:35.582655 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.582288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55506)
0807 18:24:35.582451 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.582532 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:24:35.583699 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.583310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55518)
0807 18:24:35.583456 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.583586 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:24:35.583927 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.635227 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 1
I20260807 18:24:35.636569 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.636032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55528)
0807 18:24:35.636299 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.636391 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 18:24:35.637625 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.637260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55532)
0807 18:24:35.637421 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.637516 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 18:24:35.637902 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.672622 16927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 3
I20260807 18:24:35.674041 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.673483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55538)
0807 18:24:35.673708 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.673806 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:24:35.675304 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.674846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55542)
0807 18:24:35.675097 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.675184 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260807 18:24:35.675593 16927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.706132 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 2
I20260807 18:24:35.707469 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.706890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55544)
0807 18:24:35.707162 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.707268 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 18:24:35.708720 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.708296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55554)
0807 18:24:35.708484 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.708571 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:24:35.708926 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.815147 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 3
I20260807 18:24:35.816572 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.815940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55556)
0807 18:24:35.816200 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.816318 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:24:35.817763 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.817330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55566)
0807 18:24:35.817525 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.817624 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:24:35.818019 16816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
I20260807 18:24:35.818840 16927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 4
I20260807 18:24:35.819902 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.819388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55578)
0807 18:24:35.819656 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.819742 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:24:35.821141 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.820753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55586)
0807 18:24:35.820939 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.821017 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:24:35.821458 16927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:36123: Network error: Client connection negotiation failed: client connection to 127.29.156.252:36123: connect: Connection refused (error 111)
W20260807 18:24:35.838263 16936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:35.838550 16936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:35.838759 16936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:35.846647 16936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:35.846774 16936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:35.846859 16936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:35.846956 16936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:35.847007 16936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:35.857769 16936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46161,127.29.156.253:40061,127.29.156.252:36123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:36123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.156.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:35.859133 16936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:35.860553 16936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:35.873328 16943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:35.873458 16947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:35.873685 16949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:35.874521 16936 server_base.cc:1034] running on GCE node
I20260807 18:24:35.875288 16936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:35.876811 16936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:35.878099 16936 hybrid_clock.cc:648] HybridClock initialized: now 1786127075878048 us; error 42 us; skew 500 ppm
I20260807 18:24:35.878541 16936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:35.882200 16936 init.cc:377] Logged in from keytab as kudu/127.29.156.252@KRBTEST.COM (short username kudu)
I20260807 18:24:35.884022 16936 webserver.cc:460] Webserver started at http://127.29.156.252:35903/ using document root <none> and password file <none>
I20260807 18:24:35.884796 16936 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:35.884959 16936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:35.885303 16936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:35.888568 16936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data/instance:
uuid: "4942b0418149423d8e79528138ca6491"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "34f087b4a7dfc63b14a0d5038d2115c7"
server_key_iv: "c0dae7558980126d7e494be842c0fbb3"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.889467 16936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal/instance:
uuid: "4942b0418149423d8e79528138ca6491"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "34f087b4a7dfc63b14a0d5038d2115c7"
server_key_iv: "c0dae7558980126d7e494be842c0fbb3"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.895015 16936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:24:35.899159 16959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:35.900694 16936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:24:35.900909 16936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "4942b0418149423d8e79528138ca6491"
format_stamp: "Formatted at 2026-08-07 18:24:35 on dist-test-slave-sb2z"
server_key: "34f087b4a7dfc63b14a0d5038d2115c7"
server_key_iv: "c0dae7558980126d7e494be842c0fbb3"
server_key_version: "encryptionkey@0"
I20260807 18:24:35.901132 16936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:35.923586 16936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:35.930331 16936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:35.930701 16936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:35.954909 16936 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.252:36123
I20260807 18:24:35.954972 17029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.252:36123 every 8 connection(s)
I20260807 18:24:35.957099 16936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
I20260807 18:24:35.958593 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 16936
I20260807 18:24:35.958832 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126848590636-30323-0/minicluster-data/master-2/wal/instance
I20260807 18:24:35.959447 30323 external_mini_cluster.cc:1428] Setting key 1edaad9e8df5ec113e8aff29a70b3fed
I20260807 18:24:35.962817 17030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:35.972257 17030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
Aug 07 18:24:35 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:35.990516 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.962091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.29.156.252:36123)
0807 18:24:35.963135 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.963162 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.963218 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:24:35.965737 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.965768 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.966347 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.966846 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.969057 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.971205 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.973720 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.974737 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.974979 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.980960 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:35.980972 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:35.980989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:35.981048 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:35.985286 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.986423 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.986437 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.986450 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.986726 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.987267 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.987271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.987274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.987942 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:35.988271 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:35.988794 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 18:24:35.989284 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":562,"threads_started":1}
I20260807 18:24:35.995806 16816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 4
I20260807 18:24:35.997762 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.975302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46161 (local address 127.0.0.1:50340)
0807 18:24:35.976559 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.976610 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:35.976780 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:35.977551 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:35.977562 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:35.978475 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:35.978805 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.978815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.980097 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.980108 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.981516 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.981534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.983094 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.983114 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:35.983348 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.984158 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:35.984183 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:35.986944 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:35.993702 (+ 6758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.993718 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.993729 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.994506 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.995118 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:35.995126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:35.995131 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:35.995319 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:35.996223 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:35.996235 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:35.996519 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 18:24:35.996754 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:24:35.997200 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":747,"threads_started":1}
I20260807 18:24:35.999199 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.975484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.29.156.254:46161)
0807 18:24:35.977088 (+ 1604us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.977098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.977123 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:24:35.977194 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.977202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.977301 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.977443 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.978990 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.979914 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.981706 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.982539 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.982717 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:35.987213 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:35.987223 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:35.987231 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:35.987305 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:35.993484 (+ 6179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.994699 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.994708 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.994714 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.994946 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:35.995460 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:35.995468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:35.995478 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:35.995924 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:35.996136 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:35.998351 (+ 2215us) server_negotiation.cc:299] Negotiation successful
0807 18:24:35.998627 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1450,"thread_start_us":295,"threads_started":1}
I20260807 18:24:36.001001 17030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:24:36.017107 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.996395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55604)
0807 18:24:35.996636 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:35.996655 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:35.996756 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:35.997274 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:35.997279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:35.997785 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:35.998025 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:35.998036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.999509 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:35.999517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.001151 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.001166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.002343 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.002348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.002528 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.003258 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:36.003281 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:36.006859 (+ 3578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:36.012892 (+ 6033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.012906 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.012914 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.013805 (+ 891us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.014458 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.014467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.014473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.014785 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.015679 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:36.015688 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:36.015922 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 18:24:36.016083 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 18:24:36.016480 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 18:24:36.019228 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.996447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.29.156.252:36123)
0807 18:24:35.996609 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.996615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.996702 (+ 87us) server_negotiation.cc:407] Connection header received
0807 18:24:35.996884 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.996891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.996964 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.997072 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.998210 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:35.999349 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.001327 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.002016 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.002158 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.007091 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.007098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.007116 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.007161 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.012679 (+ 5518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.014045 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.014054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.014061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.014260 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.015007 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.015014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.015018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.015465 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.018204 (+ 2739us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.018373 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.018701 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:24:36.021184 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:35.994316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.29.156.254:46161)
0807 18:24:35.995077 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:35.995083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:35.995118 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:24:35.995213 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:35.995219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:35.995289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:35.995405 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:35.998503 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.000157 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.002178 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.003258 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.003380 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.010870 (+ 7490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.010879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.010884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.010941 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.016018 (+ 5077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.017397 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.017405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.017411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.017584 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.018151 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.018159 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.018164 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.018584 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.018890 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.020526 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.020707 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":327,"threads_started":1}
I20260807 18:24:36.023871 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.002160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.29.156.253:40061)
0807 18:24:36.002371 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.002376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.002626 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:24:36.002876 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.002884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.002991 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.003068 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.004432 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.005729 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.008259 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.009136 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.009297 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.014346 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.014356 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.014364 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.014419 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.018465 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.019509 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.019515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.019519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.020319 (+ 800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.021858 (+ 1539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.021865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.021868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.022290 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.022468 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.023074 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.023324 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:24:36.025283 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.001730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:40061 (local address 127.0.0.1:35698)
0807 18:24:36.002532 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:36.002555 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:36.002726 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:36.003307 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:36.003313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:36.003884 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:36.004176 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.004191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.006122 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.006129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.007941 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.007961 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.009621 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.009631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.009916 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.010615 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:36.010653 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:36.014065 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:36.018652 (+ 4587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.018661 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.018665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.019333 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.021472 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.021477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.021481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.021690 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.022566 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:36.022571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:36.022752 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 18:24:36.024254 (+ 1502us) client_negotiation.cc:240] Negotiation successful
0807 18:24:36.024722 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664}
I20260807 18:24:36.027887 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: Bootstrap starting.
I20260807 18:24:36.034130 16816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:36.036072 16816 log.cc:824] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:36.039687 17030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:24:36.048038 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: No bootstrap required, opened a new log
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:24:36.054185 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.023457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.29.156.253:40061)
0807 18:24:36.024185 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.024194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.027944 (+ 3750us) server_negotiation.cc:407] Connection header received
0807 18:24:36.028268 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.028277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.028390 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.028504 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.031113 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.032839 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.035816 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.036907 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.037051 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.044208 (+ 7157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.044216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.044221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.044267 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.050057 (+ 5790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.051107 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.051114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.051119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.051336 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.052027 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.052037 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.052042 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.052488 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.052803 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.053459 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.053742 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":15080,"threads_started":1}
I20260807 18:24:36.061484 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:24:36.061964 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.040608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55614)
0807 18:24:36.041412 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:36.041435 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:36.041561 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:36.042143 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:36.042149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:36.042731 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:36.042985 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.043000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.044744 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.044781 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.047236 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.047256 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.049047 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.049053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.049268 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.049946 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:36.049977 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:36.053459 (+ 3482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:36.058093 (+ 4634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.058097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.058101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.058660 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.059143 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.059148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.059151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.059992 (+ 841us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.060794 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:36.060799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:36.061015 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 18:24:36.061211 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:24:36.061575 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673}
I20260807 18:24:36.062989 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.041200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.29.156.252:36123)
0807 18:24:36.041448 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.041459 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.041490 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:24:36.041731 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.041738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.041819 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.041941 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.043159 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.044431 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.047419 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.048607 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.048757 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.053666 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.053674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.053689 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.053728 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.057922 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.058806 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.058811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.058815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.059005 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.060132 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.060136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.060140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.060558 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.060723 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.062348 (+ 1625us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.062598 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:24:36.065667 16816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.066318 16816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:36.066537 16816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5783003b6fed41b3b67b30c038a6f120, State: Initialized, Role: FOLLOWER
I20260807 18:24:36.067677 16816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.074047 17030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: Bootstrap starting.
I20260807 18:24:36.075110 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.078095 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.077929 16816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:36.079902 17030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:36.081792 17030 log.cc:824] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: Log is configured to *not* fsync() on all Append() calls
W20260807 18:24:36.083828 16752 tablet.cc:2369] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:36.095538 17030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: No bootstrap required, opened a new log
I20260807 18:24:36.110034 17030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.110618 17030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:36.110776 17030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942b0418149423d8e79528138ca6491, State: Initialized, Role: FOLLOWER
I20260807 18:24:36.111717 17030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.120693 16927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } attempt: 5
I20260807 18:24:36.121783 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.095323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.29.156.254:46161)
0807 18:24:36.095652 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.095660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.095963 (+ 303us) server_negotiation.cc:407] Connection header received
0807 18:24:36.096295 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.096304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.096396 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.096502 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.099287 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.100616 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.103028 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.103893 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.104005 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.109007 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.109014 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.109019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.109072 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.113829 (+ 4757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.118349 (+ 4520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.118360 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.118367 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.118631 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.119304 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.119313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.119320 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.119854 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.120192 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.121002 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.121294 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:24:36.138008 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.096494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.29.156.253:40061)
0807 18:24:36.096741 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.096751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.098258 (+ 1507us) server_negotiation.cc:407] Connection header received
0807 18:24:36.099393 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.099403 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.099636 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.099743 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.106506 (+ 6763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.108156 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.114396 (+ 6240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.115403 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.115574 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.124550 (+ 8976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.124560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.124570 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.124629 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.130092 (+ 5463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.133174 (+ 3082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.133184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.133191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.133438 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.134626 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.134636 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.134643 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.135116 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.135407 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.137097 (+ 1690us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.137441 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:24:36.142905 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.098351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.29.156.252:36123)
0807 18:24:36.098554 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.098561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.108826 (+ 10265us) server_negotiation.cc:407] Connection header received
0807 18:24:36.115400 (+ 6574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.115407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.115507 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.115629 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.122779 (+ 7150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.124541 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.127496 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.128562 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.128704 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.133219 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.133231 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.133240 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.133299 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.138181 (+ 4882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.139391 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.139401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.139407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.139642 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.140246 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.140254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.140260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.140817 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.141147 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.141956 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.142309 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:24:36.148007 17069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.148495 17069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.148758 17069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:24:36.149086 17069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
Aug 07 18:24:36 dist-test-slave-sb2z krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:24:36.153684 17030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:36.156363 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4942b0418149423d8e79528138ca6491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5783003b6fed41b3b67b30c038a6f120" is_pre_election: true
I20260807 18:24:36.157256 16784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4942b0418149423d8e79528138ca6491 in term 0.
I20260807 18:24:36.158473 16960 leader_election.cc:302] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4942b0418149423d8e79528138ca6491, 5783003b6fed41b3b67b30c038a6f120; no voters:
I20260807 18:24:36.159130 17067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:24:36.159340 17067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:24:36.159549 17067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:36.164664 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.121372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:36123 (local address 127.0.0.1:55630)
0807 18:24:36.121796 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:36.121819 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:36.121955 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:36.143717 (+ 21762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:36.143726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:36.144326 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:36.144629 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.144647 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.146767 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.146776 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.148556 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:36.148575 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.149814 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.149820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:36.150077 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.150833 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:36.150861 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:36.154036 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:36.161293 (+ 7257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.161300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.161304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.161887 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.162435 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:36.162442 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:36.162446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:36.162667 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:36.163469 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:36.163475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:36.163684 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 18:24:36.163849 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:24:36.164190 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260807 18:24:36.164788 17069 leader_election.cc:288] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5783003b6fed41b3b67b30c038a6f120 (127.29.156.254:46161), 35a111225768484ca19f10751f3de66c (127.29.156.253:40061)
I20260807 18:24:36.163468 17067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.166102 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:36.122205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.29.156.252:36123)
0807 18:24:36.143234 (+ 21029us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:36.143243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:36.143269 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:24:36.143334 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:36.143343 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:36.143455 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:36.144827 (+ 1372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:36.144993 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.146566 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.148829 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:36.149620 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:36.154243 (+ 4623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:36.154522 (+ 279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:36.154536 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:36.154541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:36.154575 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:36.161085 (+ 6510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.162091 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.162096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.162101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.162260 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:36.162836 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:36.162841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:36.162845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:36.163273 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:36.165120 (+ 1847us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:36.165270 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:24:36.165589 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20875,"thread_start_us":22039,"threads_started":1}
I20260807 18:24:36.172690 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4942b0418149423d8e79528138ca6491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5783003b6fed41b3b67b30c038a6f120"
I20260807 18:24:36.172963 16784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:36.177752 16784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4942b0418149423d8e79528138ca6491 in term 1.
I20260807 18:24:36.179477 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: Bootstrap starting.
W20260807 18:24:36.183544 16967 tablet.cc:2369] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:36.185051 16960 leader_election.cc:302] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4942b0418149423d8e79528138ca6491, 5783003b6fed41b3b67b30c038a6f120; no voters:
I20260807 18:24:36.189800 17069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:36.190564 17069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [term 1 LEADER]: Becoming Leader. State: Replica: 4942b0418149423d8e79528138ca6491, State: Running, Role: LEADER
I20260807 18:24:36.191182 17069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.192806 16927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:36.194784 16927 log.cc:824] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:36.198798 17067 leader_election.cc:288] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [CANDIDATE]: Term 1 election: Requested vote from peers 5783003b6fed41b3b67b30c038a6f120 (127.29.156.254:46161), 35a111225768484ca19f10751f3de66c (127.29.156.253:40061)
I20260807 18:24:36.205605 17083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4942b0418149423d8e79528138ca6491. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.206095 17083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:36.207657 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: No bootstrap required, opened a new log
I20260807 18:24:36.183508 16722 master_runner.cc:186] Error getting master registration for 127.29.156.253:40061: OK, instance_id { permanent_uuid: "35a111225768484ca19f10751f3de66c" instance_seqno: 1786127075384665 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:24:36.219194 16722 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:24:36.220151 17090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260807 18:24:36.223858 17081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:24:36.224040 17081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:24:36.230777 17090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:36.251983 16927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.254350 16927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:36.254515 16927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a111225768484ca19f10751f3de66c, State: Initialized, Role: FOLLOWER
I20260807 18:24:36.253489 17105 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:24:36.258667 16927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } }
I20260807 18:24:36.267642 16784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [term 1 FOLLOWER]: Refusing update from remote peer 4942b0418149423d8e79528138ca6491: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:36.268775 17083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:36.260767 16936 master_runner.cc:186] Error getting master registration for 127.29.156.253:40061: OK, instance_id { permanent_uuid: "35a111225768484ca19f10751f3de66c" instance_seqno: 1786127075384665 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:24:36.282490 16936 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:24:36.284683 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.285778 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.286600 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4942b0418149423d8e79528138ca6491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a111225768484ca19f10751f3de66c" is_pre_election: true
I20260807 18:24:36.288955 16881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:36.293111 16881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 1 FOLLOWER]: Refusing update from remote peer 4942b0418149423d8e79528138ca6491: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:36.287999 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4942b0418149423d8e79528138ca6491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a111225768484ca19f10751f3de66c"
I20260807 18:24:36.294101 17083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:36.297758 16885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4942b0418149423d8e79528138ca6491 in term 1.
I20260807 18:24:36.298105 16927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:36.308259 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4942b0418149423d8e79528138ca6491. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.308666 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.319494 17101 mvcc.cc:205] Tried to move back new op lower bound from 7315976504377778176 to 7315976504097366016. Current Snapshot: MvccSnapshot[applied={T|T < 7315976504377778176}]
I20260807 18:24:36.328615 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.329005 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.332799 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4942b0418149423d8e79528138ca6491. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.333225 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.337292 17067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.337749 17067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:36.340730 17090 catalog_manager.cc:1347] Generated new cluster ID: d8a65ba7299f4a10b7963aeb336008b1
I20260807 18:24:36.340864 17090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:36.342933 17069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.343258 17069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:36.345232 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4942b0418149423d8e79528138ca6491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5783003b6fed41b3b67b30c038a6f120" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46161 } } peers { permanent_uuid: "35a111225768484ca19f10751f3de66c" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 40061 } } peers { permanent_uuid: "4942b0418149423d8e79528138ca6491" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 36123 } } }
I20260807 18:24:36.345520 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:24:36.368259 17090 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:24:36.370539 17090 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:36.382284 17130 catalog_manager.cc:1259] Loaded cluster ID: d8a65ba7299f4a10b7963aeb336008b1
I20260807 18:24:36.382423 17130 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: loading cluster ID for follower catalog manager: success
I20260807 18:24:36.386952 17130 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: acquiring CA information for follower catalog manager: success
W20260807 18:24:36.388655 17130 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260807 18:24:36.404903 17090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4942b0418149423d8e79528138ca6491: Generated new TSK 0
I20260807 18:24:36.406198 17090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:24:37.226172 17081 catalog_manager.cc:1259] Loaded cluster ID: d8a65ba7299f4a10b7963aeb336008b1
I20260807 18:24:37.226282 17081 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: loading cluster ID for follower catalog manager: success
I20260807 18:24:37.229025 17081 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: acquiring CA information for follower catalog manager: success
I20260807 18:24:37.230826 17081 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5783003b6fed41b3b67b30c038a6f120: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:24:37.392441 17130 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35a111225768484ca19f10751f3de66c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:24:37.555608 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 16722
I20260807 18:24:37.558187 16810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:37.670456 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 16722
I20260807 18:24:37.701197 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 16823
I20260807 18:24:37.702383 16917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:37.814973 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 16823
I20260807 18:24:37.846545 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 16936
I20260807 18:24:37.848165 17024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:37.960959 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 16936
2026-08-07T18:24:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5284 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:24:38 dist-test-slave-sb2z krb5kdc[17145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:24:38 dist-test-slave-sb2z krb5kdc[17145](info): set up 2 sockets
Aug 07 18:24:38 dist-test-slave-sb2z krb5kdc[17145](info): commencing operation
krb5kdc: starting...
W20260807 18:24:39.283951 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:24:39 dist-test-slave-sb2z krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:24:39.612181 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:42021
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:46751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:42021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:39.976785 17159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:39.977209 17159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:39.977427 17159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:39.985078 17159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:39.985198 17159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:39.985448 17159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:39.985602 17159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:39.985674 17159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:39.996623 17159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:46751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:42021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:42021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:39.998165 17159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:39.999609 17159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:40.012276 17166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:40.012319 17169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:40.012319 17167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:40.013327 17159 server_base.cc:1034] running on GCE node
I20260807 18:24:40.014230 17159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:40.015863 17159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:40.017135 17159 hybrid_clock.cc:648] HybridClock initialized: now 1786127080017069 us; error 69 us; skew 500 ppm
I20260807 18:24:40.017536 17159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:40 dist-test-slave-sb2z krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127080, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:40.025384 17159 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:40.027379 17159 webserver.cc:460] Webserver started at http://127.29.156.254:38023/ using document root <none> and password file <none>
I20260807 18:24:40.028234 17159 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:40.028410 17159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:40.028774 17159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:40.032851 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "0e9755b1d30940b4a198ca7d4553e322"
format_stamp: "Formatted at 2026-08-07 18:24:40 on dist-test-slave-sb2z"
server_key: "be0ac07ca8c01b261033f46fb9b29d6e"
server_key_iv: "c4f6a6bb507e02877ff8ba45cde8997f"
server_key_version: "encryptionkey@0"
I20260807 18:24:40.033732 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "0e9755b1d30940b4a198ca7d4553e322"
format_stamp: "Formatted at 2026-08-07 18:24:40 on dist-test-slave-sb2z"
server_key: "be0ac07ca8c01b261033f46fb9b29d6e"
server_key_iv: "c4f6a6bb507e02877ff8ba45cde8997f"
server_key_version: "encryptionkey@0"
I20260807 18:24:40.039983 17159 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:24:40.044003 17180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:40.045642 17159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:24:40.045878 17159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "0e9755b1d30940b4a198ca7d4553e322"
format_stamp: "Formatted at 2026-08-07 18:24:40 on dist-test-slave-sb2z"
server_key: "be0ac07ca8c01b261033f46fb9b29d6e"
server_key_iv: "c4f6a6bb507e02877ff8ba45cde8997f"
server_key_version: "encryptionkey@0"
I20260807 18:24:40.046113 17159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:40.067780 17159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:40.072638 17159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:40.072959 17159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:40.092839 17159 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:42021
I20260807 18:24:40.092868 17257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:42021 every 8 connection(s)
I20260807 18:24:40.094344 17159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:40.095075 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 17159
I20260807 18:24:40.095337 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:24:40.096181 30323 external_mini_cluster.cc:1428] Setting key 9420ea5682ea310c3a19de459398b744
I20260807 18:24:40.098968 17258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:24:40 dist-test-slave-sb2z krb5kdc[17145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:40.112939 17258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322: Bootstrap starting.
I20260807 18:24:40.116382 17258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:40.117535 17258 log.cc:824] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:40.120977 17258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322: No bootstrap required, opened a new log
I20260807 18:24:40.121501 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:40.098167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.29.156.254:42021)
0807 18:24:40.099060 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:40.099079 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:40.099134 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:24:40.100947 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:40.100969 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:40.101348 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:40.101700 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:40.103835 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:40.105245 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:40.107140 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:40.107918 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:40.108049 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:40.113424 (+ 5375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:40.113432 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:40.113446 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:40.113487 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:40.116917 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:40.117742 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:40.117758 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:40.117765 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:40.117953 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:40.118410 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:40.118422 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:40.118424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:40.119019 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:40.119346 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:40.119859 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 18:24:40.120366 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":399,"threads_started":1}
I20260807 18:24:40.125931 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:24:40.130577 17258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } }
I20260807 18:24:40.131003 17258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:40.131135 17258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9755b1d30940b4a198ca7d4553e322, State: Initialized, Role: FOLLOWER
I20260807 18:24:40.131825 17258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } }
I20260807 18:24:40.132085 17258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:40.132249 17258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:40.132442 17258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:40.134721 17258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } }
I20260807 18:24:40.135257 17258 leader_election.cc:302] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e9755b1d30940b4a198ca7d4553e322; no voters:
I20260807 18:24:40.135977 17258 leader_election.cc:288] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:24:40.136154 17264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:40.136966 17264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9755b1d30940b4a198ca7d4553e322, State: Running, Role: LEADER
I20260807 18:24:40.137518 17264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } }
I20260807 18:24:40.138309 17258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:40.140846 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e9755b1d30940b4a198ca7d4553e322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } } }
I20260807 18:24:40.140834 17268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e9755b1d30940b4a198ca7d4553e322. Latest consensus state: current_term: 1 leader_uuid: "0e9755b1d30940b4a198ca7d4553e322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9755b1d30940b4a198ca7d4553e322" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 42021 } } }
I20260807 18:24:40.141341 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:40.141372 17268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322 [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:40.143334 17271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:40.149216 17271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:40.158973 17271 catalog_manager.cc:1347] Generated new cluster ID: f7fe13f470bb4d82a6b0fe8614738c3c
I20260807 18:24:40.159114 17271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:40.184808 17271 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:24:40.186422 17271 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:40.202250 17271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e9755b1d30940b4a198ca7d4553e322: Generated new TSK 0
I20260807 18:24:40.203289 17271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:24:40.239008 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 17159
I20260807 18:24:40.240375 17251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:40.337651 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 17159
2026-08-07T18:24:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2371 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:24:40 dist-test-slave-sb2z krb5kdc[17298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:24:40 dist-test-slave-sb2z krb5kdc[17298](info): set up 2 sockets
Aug 07 18:24:40 dist-test-slave-sb2z krb5kdc[17298](info): commencing operation
krb5kdc: starting...
W20260807 18:24:41.708307 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:24:41 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:24:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:24:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:24:42.037755 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:42.389798 17317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:42.390125 17317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:42.390220 17317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:42.397594 17317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:42.397722 17317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:42.397771 17317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:42.397822 17317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:42.397871 17317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:42.408074 17317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:42.410013 17317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:42.411300 17317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:42.424165 17323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:42.424089 17326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:42.424199 17324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:42.424852 17317 server_base.cc:1034] running on GCE node
I20260807 18:24:42.425810 17317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:42.427249 17317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:42.428511 17317 hybrid_clock.cc:648] HybridClock initialized: now 1786127082428415 us; error 59 us; skew 500 ppm
I20260807 18:24:42.428933 17317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:42 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127082, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:42.432593 17317 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:42.434237 17317 webserver.cc:460] Webserver started at http://127.29.156.254:42537/ using document root <none> and password file <none>
I20260807 18:24:42.434986 17317 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:42.435105 17317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:42.435379 17317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:42.438716 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "7abe45c875d14f52862bd33b14ed914d"
format_stamp: "Formatted at 2026-08-07 18:24:42 on dist-test-slave-sb2z"
server_key: "765cbf9f196e6d03bec6f0b3eaf39b2b"
server_key_iv: "433fe1402489fb2ec0f74ff52b191633"
server_key_version: "encryptionkey@0"
I20260807 18:24:42.439503 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "7abe45c875d14f52862bd33b14ed914d"
format_stamp: "Formatted at 2026-08-07 18:24:42 on dist-test-slave-sb2z"
server_key: "765cbf9f196e6d03bec6f0b3eaf39b2b"
server_key_iv: "433fe1402489fb2ec0f74ff52b191633"
server_key_version: "encryptionkey@0"
I20260807 18:24:42.445359 17317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:24:42.449280 17334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:42.450765 17317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:24:42.450984 17317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "7abe45c875d14f52862bd33b14ed914d"
format_stamp: "Formatted at 2026-08-07 18:24:42 on dist-test-slave-sb2z"
server_key: "765cbf9f196e6d03bec6f0b3eaf39b2b"
server_key_iv: "433fe1402489fb2ec0f74ff52b191633"
server_key_version: "encryptionkey@0"
I20260807 18:24:42.451212 17317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:42.465767 17317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:42.470639 17317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:42.471002 17317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:42.493153 17317 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37923
I20260807 18:24:42.493202 17403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37923 every 8 connection(s)
I20260807 18:24:42.494827 17317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:42.499058 17404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:42.500113 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 17317
I20260807 18:24:42.500360 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:24:42.501302 30323 external_mini_cluster.cc:1428] Setting key 5c7695b53344472994ecda99c0d9b101
I20260807 18:24:42.513638 17404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap starting.
I20260807 18:24:42.516928 17404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Neither blocks nor log segments found. Creating new log.
Aug 07 18:24:42 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:42.518064 17404 log.cc:824] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:42.521688 17404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: No bootstrap required, opened a new log
I20260807 18:24:42.526819 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:42.503734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.29.156.254:37923)
0807 18:24:42.504809 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:42.504822 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:42.504874 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:24:42.506794 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:42.506833 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:42.507417 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:42.507999 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:42.509742 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:42.511308 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:42.513133 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:42.513948 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:42.514154 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:42.518602 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:42.518620 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:42.518645 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:42.518710 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:42.522369 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:42.523139 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:42.523155 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:42.523162 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:42.523343 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:42.523822 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:42.523828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:42.523835 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:42.524427 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:42.524797 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:42.525296 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:24:42.525749 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":577,"threads_started":1}
I20260807 18:24:42.530232 17404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:42.530627 17404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:42.530742 17404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Initialized, Role: FOLLOWER
I20260807 18:24:42.531385 17404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:42.531672 17404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:42.531841 17404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:42.532020 17404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:42.534504 17404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:42.535004 17404 leader_election.cc:302] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7abe45c875d14f52862bd33b14ed914d; no voters:
I20260807 18:24:42.535717 17404 leader_election.cc:288] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:24:42.535836 17411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:42.536659 17411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 LEADER]: Becoming Leader. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Running, Role: LEADER
I20260807 18:24:42.537166 17411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:42.537933 17404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:42.540259 17415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7abe45c875d14f52862bd33b14ed914d. Latest consensus state: current_term: 1 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:42.540844 17415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:42.541698 17413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:42.542169 17413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:42.544802 17424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:42.549871 17424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:42.558941 17424 catalog_manager.cc:1347] Generated new cluster ID: 71ffb18cdb7040b7b6666bcf3ae39cdc
I20260807 18:24:42.559074 17424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:42.581037 17424 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:24:42.583117 17424 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:42.598371 17424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Generated new TSK 0
I20260807 18:24:42.599403 17424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:24:42.689137 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:43.085405 17449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:43.085719 17449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:43.085891 17449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:43.093585 17449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:43.093747 17449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:43.093956 17449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:24:43.104452 17449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:43.105921 17449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:43.107396 17449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:43.122879 17461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:43.122838 17457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:43.123078 17458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:43.124116 17449 server_base.cc:1034] running on GCE node
I20260807 18:24:43.124948 17449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:43.126132 17449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:43.127373 17449 hybrid_clock.cc:648] HybridClock initialized: now 1786127083127324 us; error 43 us; skew 500 ppm
I20260807 18:24:43.127820 17449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:43 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127083, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:43.131386 17449 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:24:43.133224 17449 webserver.cc:460] Webserver started at http://127.29.156.193:34429/ using document root <none> and password file <none>
I20260807 18:24:43.134021 17449 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:43.134189 17449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:43.134533 17449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:43.137804 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "56fcfaf016c0463bb78e3a010ed81def"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "fee6cba6c49e19ea39aa8c85041d868f"
server_key_iv: "8dcaf31364c2ed9bf9f9e507c741f760"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.138752 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "56fcfaf016c0463bb78e3a010ed81def"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "fee6cba6c49e19ea39aa8c85041d868f"
server_key_iv: "8dcaf31364c2ed9bf9f9e507c741f760"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.144825 17449 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:24:43.149103 17472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.150777 17449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:24:43.150992 17449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "56fcfaf016c0463bb78e3a010ed81def"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "fee6cba6c49e19ea39aa8c85041d868f"
server_key_iv: "8dcaf31364c2ed9bf9f9e507c741f760"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.151239 17449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:43.192688 17449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:43.197575 17449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:43.197912 17449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:43.199030 17449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:43.200579 17449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:43.200695 17449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.200860 17449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:43.200963 17449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.240242 17449 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:43671
I20260807 18:24:43.240293 17625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:43671 every 8 connection(s)
I20260807 18:24:43.241968 17449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:24:43.248944 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 17449
I20260807 18:24:43.249208 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:24:43.249927 30323 external_mini_cluster.cc:1428] Setting key d4cce18ceeb433c01380a6af2e37aca5
Aug 07 18:24:43 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127083, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:43.264247 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:43.244813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:45243 (local address 127.29.156.254:37923)
0807 18:24:43.245339 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:43.245344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:43.246323 (+ 979us) server_negotiation.cc:407] Connection header received
0807 18:24:43.248345 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:43.248352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:43.248465 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:43.248595 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:43.250344 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.251237 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.252874 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.253270 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.253424 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:43.257826 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:43.257831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:43.257835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:43.257870 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:43.260803 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:43.261684 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:43.261688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:43.261691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:43.261815 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:43.262310 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:43.262314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:43.262316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:43.262596 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:43.262785 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:43.263623 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 18:24:43.263871 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":293,"threads_started":1}
I20260807 18:24:43.265354 17631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:43.245348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:45243)
0807 18:24:43.246139 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:43.246186 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:43.248060 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:43.248840 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:43.248849 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:43.249514 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:43.250165 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:43.250186 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.251402 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.251407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:43.252704 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:43.252717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.253521 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.253531 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:43.253778 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:43.254626 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:43.254652 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:43.257553 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:43.261000 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:43.261010 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:43.261029 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:43.261519 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:43.261938 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:43.261943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:43.261947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:43.262169 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:43.262756 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:43.262766 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:43.263158 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 18:24:43.263442 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 18:24:43.264056 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":288,"threads_started":1}
I20260807 18:24:43.267287 17627 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:43.267740 17627 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:43.268848 17627 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:43.271790 17358 ts_manager.cc:194] Registered new tserver with Master: 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671)
I20260807 18:24:43.274277 17358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:45243
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:24:43.328243 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:43.690774 17635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:43.691169 17635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:43.691362 17635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:43.698804 17635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:43.698988 17635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:43.699187 17635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:24:43.709172 17635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:43.710660 17635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:43.712122 17635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:43.726568 17648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:43.726595 17643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:43.727448 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:43.727596 17635 server_base.cc:1034] running on GCE node
I20260807 18:24:43.728308 17635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:43.729401 17635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:43.730661 17635 hybrid_clock.cc:648] HybridClock initialized: now 1786127083730609 us; error 71 us; skew 500 ppm
I20260807 18:24:43.731021 17635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:43 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127083, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:43.734719 17635 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:24:43.736433 17635 webserver.cc:460] Webserver started at http://127.29.156.194:44139/ using document root <none> and password file <none>
I20260807 18:24:43.737160 17635 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:43.737301 17635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:43.737638 17635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:43.741006 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "483c56ebca334d34869ec51e70882f4c"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "2fbb64c365741f7336ca451c4fdcea3d"
server_key_iv: "918da2ad5bb13155b124bb280ffa1876"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.741820 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "483c56ebca334d34869ec51e70882f4c"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "2fbb64c365741f7336ca451c4fdcea3d"
server_key_iv: "918da2ad5bb13155b124bb280ffa1876"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.747843 17635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:24:43.751775 17656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.753412 17635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:24:43.753641 17635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "483c56ebca334d34869ec51e70882f4c"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "2fbb64c365741f7336ca451c4fdcea3d"
server_key_iv: "918da2ad5bb13155b124bb280ffa1876"
server_key_version: "encryptionkey@0"
I20260807 18:24:43.753912 17635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:43.768532 17635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:43.773919 17635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:43.774313 17635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:43.775450 17635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:43.777021 17635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:43.777103 17635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.777245 17635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:43.777302 17635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:43.816174 17635 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43355
I20260807 18:24:43.816377 17820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43355 every 8 connection(s)
I20260807 18:24:43.817821 17635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:24:43.823153 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 17635
I20260807 18:24:43.823366 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:24:43.823891 30323 external_mini_cluster.cc:1428] Setting key 05914ee94f5e35591ce06f3665f6c017
Aug 07 18:24:43 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127083, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:43.842044 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:43.820796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:60349 (local address 127.29.156.254:37923)
0807 18:24:43.821434 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:43.821445 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:43.821994 (+ 549us) server_negotiation.cc:407] Connection header received
0807 18:24:43.824329 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:43.824335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:43.824463 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:43.824601 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:43.826493 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.827395 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.830085 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.830537 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.830652 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:43.835374 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:43.835380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:43.835383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:43.835423 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:43.838455 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:43.839397 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:43.839402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:43.839404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:43.839592 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:43.840084 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:43.840089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:43.840092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:43.840393 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:43.840557 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:43.841337 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 18:24:43.841615 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":356,"threads_started":1}
I20260807 18:24:43.843089 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:43.821024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:60349)
0807 18:24:43.821806 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:43.821847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:43.823969 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:43.824912 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:43.824925 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:43.825479 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:43.826271 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:43.826300 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.827631 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.827640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:43.829867 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:43.829886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:43.830804 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:43.830815 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:43.831186 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:43.832032 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:43.832069 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:43.835077 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:43.838643 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:43.838654 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:43.838679 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:43.839238 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:43.839720 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:43.839725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:43.839729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:43.839950 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:43.840534 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:43.840543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:43.840939 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 18:24:43.841233 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 18:24:43.841849 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":297,"threads_started":1}
I20260807 18:24:43.844795 17823 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:43.845170 17823 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:43.846112 17823 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:43.848186 17358 ts_manager.cc:194] Registered new tserver with Master: 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355)
I20260807 18:24:43.849493 17358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:60349
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:24:43.900532 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:44.265972 17835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:44.266311 17835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:44.266420 17835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:44.274811 17835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:44.274948 17835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:44.275074 17835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:24:44.278805 17627 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:44.286125 17835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:44.287618 17835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:44.289111 17835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:44.303133 17842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:44.303144 17843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:44.303162 17849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:44.305294 17835 server_base.cc:1034] running on GCE node
I20260807 18:24:44.306028 17835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:44.307144 17835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:44.308408 17835 hybrid_clock.cc:648] HybridClock initialized: now 1786127084308356 us; error 71 us; skew 500 ppm
I20260807 18:24:44.308769 17835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:44 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127084, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:44.312202 17835 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:24:44.313946 17835 webserver.cc:460] Webserver started at http://127.29.156.195:45285/ using document root <none> and password file <none>
I20260807 18:24:44.314780 17835 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:44.314939 17835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:44.315268 17835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:24:44.318590 17835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "2890f67badf9486bb3eca41b24c6316e"
format_stamp: "Formatted at 2026-08-07 18:24:44 on dist-test-slave-sb2z"
server_key: "238bb83b53ff0612447a6bf8cb70a01e"
server_key_iv: "46713f5ea09d8a2fcb379482dfc4768a"
server_key_version: "encryptionkey@0"
I20260807 18:24:44.319456 17835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "2890f67badf9486bb3eca41b24c6316e"
format_stamp: "Formatted at 2026-08-07 18:24:44 on dist-test-slave-sb2z"
server_key: "238bb83b53ff0612447a6bf8cb70a01e"
server_key_iv: "46713f5ea09d8a2fcb379482dfc4768a"
server_key_version: "encryptionkey@0"
I20260807 18:24:44.325136 17835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:24:44.328919 17858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:44.330399 17835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:24:44.330600 17835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "2890f67badf9486bb3eca41b24c6316e"
format_stamp: "Formatted at 2026-08-07 18:24:44 on dist-test-slave-sb2z"
server_key: "238bb83b53ff0612447a6bf8cb70a01e"
server_key_iv: "46713f5ea09d8a2fcb379482dfc4768a"
server_key_version: "encryptionkey@0"
I20260807 18:24:44.330818 17835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:44.351305 17835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:44.356307 17835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:44.356659 17835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:44.357688 17835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:44.359366 17835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:44.359463 17835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:44.359661 17835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:44.359735 17835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:44.396955 17835 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33117
I20260807 18:24:44.397063 18016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33117 every 8 connection(s)
I20260807 18:24:44.398362 17835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:24:44.404418 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 17835
I20260807 18:24:44.404627 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:24:44.405169 30323 external_mini_cluster.cc:1428] Setting key 09a1921179d52c386e5041d2e15a8a34
Aug 07 18:24:44 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127084, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:44.422220 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.401374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:46221 (local address 127.29.156.254:37923)
0807 18:24:44.401860 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.401867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.403095 (+ 1228us) server_negotiation.cc:407] Connection header received
0807 18:24:44.405255 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.405261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.405379 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.405543 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:44.407664 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.408783 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.410540 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.411023 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.411144 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.415091 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:44.415097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:44.415100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:44.415149 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:44.418167 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:44.418996 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:44.419001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:44.419003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:44.419208 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:44.419773 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:44.419777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:44.419779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:44.420227 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:44.420429 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.421430 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.421747 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":220,"threads_started":1}
I20260807 18:24:44.423197 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.402089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:46221)
0807 18:24:44.402901 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.402957 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.404925 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.405782 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.405798 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.406600 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:44.407411 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.407441 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.409035 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.409039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.410345 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.410358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.411285 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.411291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.411519 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.412376 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:44.412405 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:44.414802 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:44.418335 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:44.418342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:44.418357 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:44.418801 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:44.419337 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:44.419341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:44.419343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:44.419635 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:44.420381 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:44.420409 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:44.420871 (+ 462us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.421314 (+ 443us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.421951 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":330,"threads_started":1}
I20260807 18:24:44.424718 18017 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:44.425011 18017 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:44.425745 18017 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:44.427301 17358 ts_manager.cc:194] Registered new tserver with Master: 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117)
I20260807 18:24:44.428260 17358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:46221
I20260807 18:24:44.434820 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:24:44.455211 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.440281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.29.156.254:37923)
0807 18:24:44.440582 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.440591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.440899 (+ 308us) server_negotiation.cc:407] Connection header received
0807 18:24:44.441210 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.441214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.441287 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.441402 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:44.442953 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.443789 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.445257 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.445719 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.445867 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.448872 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:44.448878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:44.448881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:44.448922 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:44.452204 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:44.452968 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:44.452974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:44.452978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:44.453123 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:44.453551 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:44.453559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:44.453563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:44.453918 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:44.454214 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.454581 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.454843 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:24:44.460310 17358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:24:44.467337 17358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:24:44.500782 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.483103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33117 (local address 127.0.0.1:50408)
0807 18:24:44.489514 (+ 6411us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.489534 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.489652 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.490962 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.490972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.491005 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:44.491496 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.491511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.493838 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.493843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.496873 (+ 3030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.496890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.499466 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.499474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.499791 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.499873 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.500035 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.500256 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6253,"mutex_wait_us":37,"thread_start_us":6095,"threads_started":1}
I20260807 18:24:44.501884 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.483905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50408 (local address 127.29.156.195:33117)
0807 18:24:44.490355 (+ 6450us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.490363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.490397 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:24:44.490485 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.490491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.490713 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.490962 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:44.491803 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.493655 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.497092 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.498832 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.499013 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.499221 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.501257 (+ 2036us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.501480 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6267,"thread_start_us":6117,"threads_started":1}
I20260807 18:24:44.502173 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.481651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43671 (local address 127.0.0.1:38706)
0807 18:24:44.488612 (+ 6961us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.488684 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.488977 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.493100 (+ 4123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.493105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.493133 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:44.493793 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.493806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.496346 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.496352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.499008 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.499025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.501227 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.501235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.501441 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.501479 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.501548 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.501690 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":1,"spinlock_wait_cycles":2304,"thread_start_us":7157,"threads_started":1}
I20260807 18:24:44.503700 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.482270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.29.156.193:43671)
0807 18:24:44.492314 (+ 10044us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.492325 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.492357 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:24:44.492463 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.492471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.492817 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.494040 (+ 1223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:44.494300 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.496173 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.499267 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.500929 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.502827 (+ 1898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.503044 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.503117 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.503298 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9824,"thread_start_us":9621,"threads_started":1}
I20260807 18:24:44.506381 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.481733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43355 (local address 127.0.0.1:41524)
0807 18:24:44.489226 (+ 7493us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.489257 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.489451 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.493192 (+ 3741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.493203 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.493248 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:44.494885 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.494901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.498582 (+ 3681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.498588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.501235 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.501250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.505389 (+ 4139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.505396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.505599 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.505635 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.505751 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.505913 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7292,"mutex_wait_us":1038,"spinlock_wait_cycles":5120,"thread_start_us":2588,"threads_started":1}
I20260807 18:24:44.507134 17929 tablet_service.cc:1467] Processing CreateTablet for tablet 00de043b6829421bb8deb72b01882b59 (DEFAULT_TABLE table=test-table [id=2a8b402dcebd4590bda88f40a0c25c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:44.507417 17539 tablet_service.cc:1467] Processing CreateTablet for tablet 00de043b6829421bb8deb72b01882b59 (DEFAULT_TABLE table=test-table [id=2a8b402dcebd4590bda88f40a0c25c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:44.508620 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.482307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.29.156.194:43355)
0807 18:24:44.491761 (+ 9454us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.491772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.492299 (+ 527us) server_negotiation.cc:407] Connection header received
0807 18:24:44.492418 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.492426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.492689 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.492892 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:44.495145 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.498354 (+ 3209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.503014 (+ 4660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.504788 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.505035 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.505218 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.506639 (+ 1421us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.508259 (+ 1620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9245,"thread_start_us":9096,"threads_started":1}
I20260807 18:24:44.509214 17929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00de043b6829421bb8deb72b01882b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:44.509563 17539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00de043b6829421bb8deb72b01882b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:44.512962 17723 tablet_service.cc:1467] Processing CreateTablet for tablet 00de043b6829421bb8deb72b01882b59 (DEFAULT_TABLE table=test-table [id=2a8b402dcebd4590bda88f40a0c25c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:44.514645 17723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00de043b6829421bb8deb72b01882b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:44.526594 18046 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Bootstrap starting.
I20260807 18:24:44.529098 18047 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Bootstrap starting.
I20260807 18:24:44.531283 18049 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Bootstrap starting.
I20260807 18:24:44.532478 18046 tablet_bootstrap.cc:654] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:44.533493 18047 tablet_bootstrap.cc:654] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:44.534004 18046 log.cc:824] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:44.534869 18049 tablet_bootstrap.cc:654] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:44.535041 18047 log.cc:824] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:44.536388 18049 log.cc:824] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:44.538208 18046 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: No bootstrap required, opened a new log
I20260807 18:24:44.538568 18046 ts_tablet_manager.cc:1397] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:24:44.539022 18047 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: No bootstrap required, opened a new log
I20260807 18:24:44.539402 18047 ts_tablet_manager.cc:1397] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260807 18:24:44.539798 18049 tablet_bootstrap.cc:492] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: No bootstrap required, opened a new log
I20260807 18:24:44.540096 18049 ts_tablet_manager.cc:1397] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 18:24:44.547897 18046 raft_consensus.cc:348] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.548436 18046 raft_consensus.cc:374] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:44.548573 18046 raft_consensus.cc:729] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2890f67badf9486bb3eca41b24c6316e, State: Initialized, Role: FOLLOWER
I20260807 18:24:44.549341 18046 consensus_queue.cc:260] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.550693 18017 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:44.551340 18046 ts_tablet_manager.cc:1428] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260807 18:24:44.551712 18047 raft_consensus.cc:348] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.552243 18047 raft_consensus.cc:374] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:44.552377 18047 raft_consensus.cc:729] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fcfaf016c0463bb78e3a010ed81def, State: Initialized, Role: FOLLOWER
I20260807 18:24:44.553189 18047 consensus_queue.cc:260] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.554819 18049 raft_consensus.cc:348] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.555366 18049 raft_consensus.cc:374] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:44.555420 18047 ts_tablet_manager.cc:1428] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260807 18:24:44.555497 18049 raft_consensus.cc:729] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483c56ebca334d34869ec51e70882f4c, State: Initialized, Role: FOLLOWER
I20260807 18:24:44.556377 18049 consensus_queue.cc:260] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.557902 17823 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:44.558917 18049 ts_tablet_manager.cc:1428] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
W20260807 18:24:44.570218 17824 tablet.cc:2369] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:24:44.650292 18018 tablet.cc:2369] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:24:44.744912 17628 tablet.cc:2369] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:44.861395 18055 raft_consensus.cc:482] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:24:44.861792 18055 raft_consensus.cc:504] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.864250 18055 leader_election.cc:288] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671), 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355)
I20260807 18:24:44.874962 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.864750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43355 (local address 127.29.156.195:53185)
0807 18:24:44.865295 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.865319 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.865477 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.866548 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.866556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.866606 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:44.866859 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.866873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.868707 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.868718 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.871760 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.871782 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.873920 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.873926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.874169 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.874233 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.874357 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.874548 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":298}
I20260807 18:24:44.875763 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.864750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:53185 (local address 127.29.156.194:43355)
0807 18:24:44.865001 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.865007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.866029 (+ 1022us) server_negotiation.cc:407] Connection header received
0807 18:24:44.866133 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.866141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.866243 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.866346 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:44.867076 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.868443 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.871997 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.873702 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.873894 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.874005 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.875247 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.875401 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:24:44.877013 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00de043b6829421bb8deb72b01882b59" candidate_uuid: "2890f67badf9486bb3eca41b24c6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483c56ebca334d34869ec51e70882f4c" is_pre_election: true
I20260807 18:24:44.877787 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.865901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:36655 (local address 127.29.156.193:43671)
0807 18:24:44.866158 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.866165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.866186 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:24:44.866264 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.866269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.866351 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.866452 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:44.867210 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.868419 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.875411 (+ 6992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.876647 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.876891 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.876970 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.877315 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.877487 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:24:44.877760 17751 raft_consensus.cc:2454] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2890f67badf9486bb3eca41b24c6316e in term 0.
I20260807 18:24:44.878544 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.864778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43671 (local address 127.29.156.195:36655)
0807 18:24:44.865414 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:44.865432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:44.865583 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:44.866473 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:44.866478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:44.866504 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:44.867035 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.867048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.868592 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.868599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.875025 (+ 6426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:44.875044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.876922 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.876928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:44.877099 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.877131 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:24:44.877982 (+ 851us) client_negotiation.cc:240] Negotiation successful
0807 18:24:44.878135 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":3363,"spinlock_wait_cycles":24320,"thread_start_us":9418,"threads_started":1}
I20260807 18:24:44.878885 17859 leader_election.cc:302] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2890f67badf9486bb3eca41b24c6316e, 483c56ebca334d34869ec51e70882f4c; no voters:
I20260807 18:24:44.879415 18055 raft_consensus.cc:2781] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:24:44.879624 18055 raft_consensus.cc:482] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:24:44.879750 18055 raft_consensus.cc:3037] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:44.879884 17563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00de043b6829421bb8deb72b01882b59" candidate_uuid: "2890f67badf9486bb3eca41b24c6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fcfaf016c0463bb78e3a010ed81def" is_pre_election: true
I20260807 18:24:44.880483 17563 raft_consensus.cc:2454] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2890f67badf9486bb3eca41b24c6316e in term 0.
I20260807 18:24:44.883502 18055 raft_consensus.cc:504] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.884470 18055 leader_election.cc:288] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [CANDIDATE]: Term 1 election: Requested vote from peers 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671), 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355)
I20260807 18:24:44.885012 17563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00de043b6829421bb8deb72b01882b59" candidate_uuid: "2890f67badf9486bb3eca41b24c6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fcfaf016c0463bb78e3a010ed81def"
I20260807 18:24:44.884985 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00de043b6829421bb8deb72b01882b59" candidate_uuid: "2890f67badf9486bb3eca41b24c6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483c56ebca334d34869ec51e70882f4c"
I20260807 18:24:44.885264 17563 raft_consensus.cc:3037] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:44.885295 17751 raft_consensus.cc:3037] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:44.889178 17563 raft_consensus.cc:2454] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2890f67badf9486bb3eca41b24c6316e in term 1.
I20260807 18:24:44.889293 17751 raft_consensus.cc:2454] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2890f67badf9486bb3eca41b24c6316e in term 1.
I20260807 18:24:44.889801 17859 leader_election.cc:302] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2890f67badf9486bb3eca41b24c6316e, 483c56ebca334d34869ec51e70882f4c; no voters:
I20260807 18:24:44.890161 18055 raft_consensus.cc:2781] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:44.890774 18055 raft_consensus.cc:686] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 1 LEADER]: Becoming Leader. State: Replica: 2890f67badf9486bb3eca41b24c6316e, State: Running, Role: LEADER
I20260807 18:24:44.891431 18055 consensus_queue.cc:237] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:44.897902 17358 catalog_manager.cc:5495] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2890f67badf9486bb3eca41b24c6316e (127.29.156.195). New cstate: current_term: 1 leader_uuid: "2890f67badf9486bb3eca41b24c6316e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } health_report { overall_health: HEALTHY } } }
I20260807 18:24:44.978444 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:44.970837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50420 (local address 127.29.156.195:33117)
0807 18:24:44.971131 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:44.971141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:44.971397 (+ 256us) server_negotiation.cc:407] Connection header received
0807 18:24:44.971644 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:44.971652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:44.971795 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:44.972011 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:24:44.972411 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.973716 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.975277 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:44.976119 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:44.976270 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:44.976818 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:24:44.977683 (+ 865us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:24:44.977858 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:44.977913 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:24:44.978060 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:24:44.989051 17751 raft_consensus.cc:1261] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Refusing update from remote peer 2890f67badf9486bb3eca41b24c6316e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:44.989218 17563 raft_consensus.cc:1261] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Refusing update from remote peer 2890f67badf9486bb3eca41b24c6316e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:44.990275 18062 consensus_queue.cc:1035] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [LEADER]: Connected to new peer: Peer: permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:44.990837 18055 consensus_queue.cc:1035] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:45.015997 18071 mvcc.cc:205] Tried to move back new op lower bound from 7315976540103401472 to 7315976539721117696. Current Snapshot: MvccSnapshot[applied={T|T < 7315976540103401472}]
I20260807 18:24:45.043905 17358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:45.044252 17358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:45.048925 17358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Sending DeleteTablet for 3 replicas of tablet 00de043b6829421bb8deb72b01882b59
I20260807 18:24:45.050540 17723 tablet_service.cc:1514] Processing DeleteTablet for tablet 00de043b6829421bb8deb72b01882b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:45 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:41524
I20260807 18:24:45.050601 17539 tablet_service.cc:1514] Processing DeleteTablet for tablet 00de043b6829421bb8deb72b01882b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:45 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:38706
I20260807 18:24:45.052675 17929 tablet_service.cc:1514] Processing DeleteTablet for tablet 00de043b6829421bb8deb72b01882b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:45 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:50408
I20260807 18:24:45.057138 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 17449
I20260807 18:24:45.089059 18090 tablet_replica.cc:330] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: stopping tablet replica
I20260807 18:24:45.089828 18090 raft_consensus.cc:2229] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:24:45.090998 18090 raft_consensus.cc:2258] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:45.093799 18090 ts_tablet_manager.cc:1905] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:45.099157 18089 tablet_replica.cc:330] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: stopping tablet replica
I20260807 18:24:45.099969 18089 raft_consensus.cc:2229] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:45.104118 18089 raft_consensus.cc:2258] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:45.104789 18085 tablet_replica.cc:330] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: stopping tablet replica
I20260807 18:24:45.105360 18085 raft_consensus.cc:2229] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:45.105859 18085 raft_consensus.cc:2258] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:45.108578 18089 ts_tablet_manager.cc:1905] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:45.113170 18085 ts_tablet_manager.cc:1905] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:45.118160 18089 ts_tablet_manager.cc:1918] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:45.118495 18089 log.cc:1196] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/00de043b6829421bb8deb72b01882b59
I20260807 18:24:45.119259 18089 ts_tablet_manager.cc:1939] T 00de043b6829421bb8deb72b01882b59 P 56fcfaf016c0463bb78e3a010ed81def: Deleting consensus metadata
I20260807 18:24:45.121970 17343 catalog_manager.cc:4841] TS 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671): tablet 00de043b6829421bb8deb72b01882b59 (table test-table [id=2a8b402dcebd4590bda88f40a0c25c84]) successfully deleted
I20260807 18:24:45.122334 18090 ts_tablet_manager.cc:1918] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 18:24:45 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:24:45.131750 18085 ts_tablet_manager.cc:1918] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:45.131940 18085 log.cc:1196] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/00de043b6829421bb8deb72b01882b59
I20260807 18:24:45.132547 18085 ts_tablet_manager.cc:1939] T 00de043b6829421bb8deb72b01882b59 P 483c56ebca334d34869ec51e70882f4c: Deleting consensus metadata
I20260807 18:24:45.134549 17336 catalog_manager.cc:4841] TS 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355): tablet 00de043b6829421bb8deb72b01882b59 (table test-table [id=2a8b402dcebd4590bda88f40a0c25c84]) successfully deleted
I20260807 18:24:45.134611 18090 log.cc:1196] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/00de043b6829421bb8deb72b01882b59
I20260807 18:24:45.135284 18090 ts_tablet_manager.cc:1939] T 00de043b6829421bb8deb72b01882b59 P 2890f67badf9486bb3eca41b24c6316e: Deleting consensus metadata
I20260807 18:24:45.135516 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:45.058572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.29.156.193:43671)
0807 18:24:45.058792 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:45.058801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:45.094779 (+ 35978us) server_negotiation.cc:407] Connection header received
0807 18:24:45.095178 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:45.095187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:45.095294 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:45.095387 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:45.100446 (+ 5059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.101855 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.115052 (+ 13197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.115910 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.116051 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:45.126476 (+ 10425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:45.126494 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:45.126514 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:45.126577 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:45.130648 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.131769 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.131779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.131783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.132139 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.132681 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.132687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.132691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.133266 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:45.133566 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:45.134185 (+ 619us) server_negotiation.cc:299] Negotiation successful
0807 18:24:45.134957 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:24:45.137351 17342 catalog_manager.cc:4841] TS 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117): tablet 00de043b6829421bb8deb72b01882b59 (table test-table [id=2a8b402dcebd4590bda88f40a0c25c84]) successfully deleted
I20260807 18:24:45.138612 17621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:45.238613 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 17449
I20260807 18:24:45.270982 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 17635
Aug 07 18:24:45 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:24:45.290414 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:45.272023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.29.156.194:43355)
0807 18:24:45.272252 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:45.272256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:45.272276 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:24:45.272468 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:45.272472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:45.272537 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:45.272628 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:45.274479 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.275286 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.277374 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.277784 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.277896 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:45.283655 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:45.283682 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:45.283699 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:45.283746 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:45.287132 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.288049 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.288057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.288059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.288208 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.288651 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.288655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.288657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.289069 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:45.289423 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:45.289788 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:24:45.290039 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 18:24:45.291283 17814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:45.388387 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 17635
I20260807 18:24:45.423650 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 17835
Aug 07 18:24:45 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:24:45.444152 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:45.424781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.29.156.195:33117)
0807 18:24:45.425018 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:45.425025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:45.425049 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:24:45.425128 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:45.425134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:45.425247 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:45.425374 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:45.426754 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.428002 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.429523 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:45.430225 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:45.430394 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:45.435863 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:45.435886 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:45.435914 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:45.435980 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:45.440607 (+ 4627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.441457 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.441469 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.441472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.441705 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:45.442173 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:45.442179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:45.442183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:45.442701 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:45.443055 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:45.443285 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 18:24:45.443694 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:24:45.445043 18010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:45.575971 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 17835
I20260807 18:24:45.604786 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 17317
I20260807 18:24:45.606078 17398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:45.720463 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 17317
I20260807 18:24:45.751969 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--webserver_interface=127.29.156.254
--webserver_port=42537
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:46.106577 18096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:46.106884 18096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:46.107057 18096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:46.114324 18096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:46.114437 18096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:46.114600 18096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:46.114712 18096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:46.114784 18096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:46.124823 18096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=42537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:46.126192 18096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:46.127575 18096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:46.154510 18105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:46.154577 18107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:46.154582 18104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:46.155638 18096 server_base.cc:1034] running on GCE node
I20260807 18:24:46.156728 18096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:46.158954 18096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:46.160209 18096 hybrid_clock.cc:648] HybridClock initialized: now 1786127086160153 us; error 50 us; skew 500 ppm
I20260807 18:24:46.160687 18096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:46 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127086, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:46.164664 18096 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:46.166371 18096 webserver.cc:460] Webserver started at http://127.29.156.254:42537/ using document root <none> and password file <none>
I20260807 18:24:46.167200 18096 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:46.167371 18096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:46.173780 18096 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:24:46.177174 18121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:46.178658 18096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:24:46.178866 18096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "7abe45c875d14f52862bd33b14ed914d"
format_stamp: "Formatted at 2026-08-07 18:24:42 on dist-test-slave-sb2z"
server_key: "765cbf9f196e6d03bec6f0b3eaf39b2b"
server_key_iv: "433fe1402489fb2ec0f74ff52b191633"
server_key_version: "encryptionkey@0"
I20260807 18:24:46.180140 18096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:46.198156 18096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:46.202734 18096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:46.203038 18096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:46.223073 18096 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37923
I20260807 18:24:46.223080 18195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37923 every 8 connection(s)
I20260807 18:24:46.224706 18096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:46.231815 18196 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:24:46.234699 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18096
I20260807 18:24:46.235265 18196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap starting.
I20260807 18:24:46.235659 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:43671
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=34429
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:46.257076 18196 log.cc:824] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:46.269622 18196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:24:46.270205 18196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap complete.
I20260807 18:24:46.279861 18196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:46.280745 18196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Initialized, Role: FOLLOWER
I20260807 18:24:46.281462 18196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:46.281721 18196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:46.281853 18196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:46.282052 18196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:24:46.286499 18196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:46.287058 18196 leader_election.cc:302] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7abe45c875d14f52862bd33b14ed914d; no voters:
I20260807 18:24:46.287719 18196 leader_election.cc:288] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:24:46.287873 18203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:24:46.289187 18203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 LEADER]: Becoming Leader. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Running, Role: LEADER
I20260807 18:24:46.289834 18203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:46.290226 18196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:46.292577 18205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:46.292970 18205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:46.293002 18208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7abe45c875d14f52862bd33b14ed914d. Latest consensus state: current_term: 2 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:46.293426 18208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:46.307952 18215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:46.316298 18215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:46.317817 18215 catalog_manager.cc:1259] Loaded cluster ID: 71ffb18cdb7040b7b6666bcf3ae39cdc
I20260807 18:24:46.317989 18215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:24:46.321310 18215 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:24:46.339272 18215 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:24:46.349118 18215 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:24:46.351001 18215 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Loaded TSK: 0
I20260807 18:24:46.352584 18215 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:24:46.616633 18198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:46.616937 18198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:46.617100 18198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:46.624562 18198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:46.624670 18198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:46.624840 18198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:24:46.634605 18198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=34429
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:46.636001 18198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:46.637454 18198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:46.652558 18241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:46.652534 18242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:46.653548 18244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:46.653862 18198 server_base.cc:1034] running on GCE node
I20260807 18:24:46.654578 18198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:46.655681 18198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:46.656972 18198 hybrid_clock.cc:648] HybridClock initialized: now 1786127086656921 us; error 31 us; skew 500 ppm
I20260807 18:24:46.657419 18198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:46 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127086, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:46.661626 18198 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:24:46.663391 18198 webserver.cc:460] Webserver started at http://127.29.156.193:34429/ using document root <none> and password file <none>
I20260807 18:24:46.664218 18198 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:46.664469 18198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:46.670723 18198 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:24:46.674042 18254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:46.675689 18198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:24:46.675885 18198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "56fcfaf016c0463bb78e3a010ed81def"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "fee6cba6c49e19ea39aa8c85041d868f"
server_key_iv: "8dcaf31364c2ed9bf9f9e507c741f760"
server_key_version: "encryptionkey@0"
I20260807 18:24:46.677170 18198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:46.694520 18198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:46.698944 18198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:46.699249 18198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:46.700373 18198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:46.701797 18198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:46.701932 18198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:46.702077 18198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:46.702175 18198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:46.739176 18198 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:43671
I20260807 18:24:46.739267 18413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:43671 every 8 connection(s)
I20260807 18:24:46.740734 18198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:24:46.748499 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18198
I20260807 18:24:46.749226 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:43355
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=44139
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:46 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127086, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:46.765442 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:46.744324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:35257)
0807 18:24:46.745229 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:46.745285 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:46.747048 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:46.748539 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:46.748580 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:46.749244 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:46.749790 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:46.749808 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:46.751467 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:46.751474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:46.752837 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:46.752849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:46.753899 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:46.753907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:46.754182 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:46.754896 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:46.754920 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:46.757211 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:46.761266 (+ 4055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:46.761273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:46.761284 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:46.761724 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:46.762269 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:46.762272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:46.762274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:46.762462 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:46.763430 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:46.763436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:46.763823 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 18:24:46.764046 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 18:24:46.764436 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":397,"threads_started":1}
I20260807 18:24:46.766111 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:46.744033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:35257 (local address 127.29.156.254:37923)
0807 18:24:46.744648 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:46.744661 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:46.745439 (+ 778us) server_negotiation.cc:407] Connection header received
0807 18:24:46.747634 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:46.747647 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:46.747990 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:46.748393 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:46.750029 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:46.751287 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:46.752990 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:46.753650 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:46.753862 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:46.757489 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:46.757497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:46.757511 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:46.757548 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:46.761101 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:46.761885 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:46.761894 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:46.761903 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:46.762148 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:46.762588 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:46.762596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:46.762601 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:46.763282 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:46.763484 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:46.764345 (+ 861us) server_negotiation.cc:299] Negotiation successful
0807 18:24:46.764809 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":242,"threads_started":1}
I20260807 18:24:46.768975 18415 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:46.769320 18415 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:46.770287 18415 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:46.772823 18149 ts_manager.cc:194] Registered new tserver with Master: 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671)
I20260807 18:24:46.774507 18149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:35257
W20260807 18:24:47.107472 18421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:47.107874 18421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:47.108045 18421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:47.115733 18421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:47.115890 18421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:47.116092 18421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:24:47.126217 18421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:43355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=44139
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:47.127676 18421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:47.129107 18421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:47.143039 18434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:47.143088 18431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:47.143107 18429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:47.143867 18421 server_base.cc:1034] running on GCE node
I20260807 18:24:47.144519 18421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:47.145475 18421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:47.146745 18421 hybrid_clock.cc:648] HybridClock initialized: now 1786127087146695 us; error 58 us; skew 500 ppm
I20260807 18:24:47.147153 18421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:47 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:47.150475 18421 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:24:47.152088 18421 webserver.cc:460] Webserver started at http://127.29.156.194:44139/ using document root <none> and password file <none>
I20260807 18:24:47.152751 18421 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:47.152876 18421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:47.158478 18421 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:24:47.161602 18446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.163033 18421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:24:47.163172 18421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "483c56ebca334d34869ec51e70882f4c"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "2fbb64c365741f7336ca451c4fdcea3d"
server_key_iv: "918da2ad5bb13155b124bb280ffa1876"
server_key_version: "encryptionkey@0"
I20260807 18:24:47.164292 18421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:47.178463 18421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:47.183094 18421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:47.183421 18421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:47.184544 18421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:47.185931 18421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:47.186036 18421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.186118 18421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:47.186175 18421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.221937 18421 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43355
I20260807 18:24:47.222090 18600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43355 every 8 connection(s)
I20260807 18:24:47.223876 18421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:24:47.224417 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18421
I20260807 18:24:47.225296 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:33117
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=45285
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:24:47 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:47.250636 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.228404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:58361 (local address 127.29.156.254:37923)
0807 18:24:47.228645 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.228654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.230308 (+ 1654us) server_negotiation.cc:407] Connection header received
0807 18:24:47.233765 (+ 3457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.233773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.233894 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.234013 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:47.236031 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.237206 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.239178 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.239711 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.239858 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.243883 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:47.243892 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:47.243898 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:47.243941 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:47.247429 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.248223 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.248229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.248232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.248357 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.248813 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.248822 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.248824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.249139 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:47.249307 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.250026 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.250276 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:24:47.251960 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.227781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:58361)
0807 18:24:47.230143 (+ 2362us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.230191 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.233417 (+ 3226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.234289 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.234301 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.235015 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:47.235824 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.235849 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.237412 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.237420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.238985 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.239003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.239963 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.239970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.240213 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.240945 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:47.240968 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:47.243606 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:47.247620 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:47.247628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:47.247641 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:47.248083 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:47.248474 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:47.248479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:47.248484 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:47.248681 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:47.249277 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:47.249284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:47.249637 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.249908 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.250467 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2049,"thread_start_us":1837,"threads_started":1}
I20260807 18:24:47.253579 18601 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:47.253961 18601 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:47.254778 18601 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:47.256301 18149 ts_manager.cc:194] Registered new tserver with Master: 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355)
I20260807 18:24:47.257123 18149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:58361
W20260807 18:24:47.581205 18604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:47.581583 18604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:47.581725 18604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:47.588997 18604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:47.589113 18604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:47.589229 18604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:24:47.598791 18604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:33117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=45285
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:47.600057 18604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:47.601497 18604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:47.615096 18614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:47.615146 18618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:47.615175 18615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:47.616474 18604 server_base.cc:1034] running on GCE node
I20260807 18:24:47.617286 18604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:47.618351 18604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:47.619653 18604 hybrid_clock.cc:648] HybridClock initialized: now 1786127087619589 us; error 53 us; skew 500 ppm
I20260807 18:24:47.620070 18604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:47 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:47.623766 18604 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:24:47.625335 18604 webserver.cc:460] Webserver started at http://127.29.156.195:45285/ using document root <none> and password file <none>
I20260807 18:24:47.626084 18604 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:47.626219 18604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:47.632563 18604 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:24:47.635349 18630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.636889 18604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:24:47.637094 18604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "2890f67badf9486bb3eca41b24c6316e"
format_stamp: "Formatted at 2026-08-07 18:24:44 on dist-test-slave-sb2z"
server_key: "238bb83b53ff0612447a6bf8cb70a01e"
server_key_iv: "46713f5ea09d8a2fcb379482dfc4768a"
server_key_version: "encryptionkey@0"
I20260807 18:24:47.638134 18604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:47.671370 18604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:47.676062 18604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:47.676301 18604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:47.677311 18604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:47.678663 18604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:47.678755 18604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.678860 18604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:47.678997 18604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:47.714874 18604 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33117
I20260807 18:24:47.715054 18773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33117 every 8 connection(s)
I20260807 18:24:47.716570 18604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:24:47.723232 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18604
Aug 07 18:24:47 dist-test-slave-sb2z krb5kdc[17298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:24:47.739391 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.719431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:59201 (local address 127.29.156.254:37923)
0807 18:24:47.719697 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.719703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.720943 (+ 1240us) server_negotiation.cc:407] Connection header received
0807 18:24:47.722822 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.722829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.722925 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.723061 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:47.724898 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.726013 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.727912 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.728397 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.728548 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.732497 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:47.732506 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:47.732513 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:47.732561 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:47.735724 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.736579 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.736587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.736589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.736722 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.737268 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.737272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.737274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.737534 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:47.737704 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.738842 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.739047 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 18:24:47.740397 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.720033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:59201)
0807 18:24:47.720756 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.720803 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.722484 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.723274 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.723282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.723904 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:24:47.724660 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.724688 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.726208 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.726212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.727769 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.727786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.728617 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.728623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.728836 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.729546 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:24:47.729569 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:24:47.732238 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:24:47.735879 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:47.735888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:47.735906 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:47.736440 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:47.736847 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:24:47.736852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:24:47.736856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:24:47.737087 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:24:47.737677 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:24:47.737685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:24:47.738025 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.738326 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.738899 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":281,"threads_started":1}
I20260807 18:24:47.741582 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.724302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.29.156.254:37923)
0807 18:24:47.725261 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.725265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.725280 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:24:47.725319 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.725322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.725376 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.725466 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:47.727229 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.728286 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.729807 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.730489 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.730636 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.733910 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:47.733915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:47.733918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:47.733964 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:47.737042 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.738023 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.738030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.738033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.738166 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:47.738682 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:47.738686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:47.738689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:47.739034 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:47.739283 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.740281 (+ 998us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.740508 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":692,"threads_started":1}
I20260807 18:24:47.742612 18774 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37923
I20260807 18:24:47.742969 18774 heartbeater.cc:464] Registering TS with master...
I20260807 18:24:47.743831 18774 heartbeater.cc:511] Master 127.29.156.254:37923 requested a full tablet report, sending...
I20260807 18:24:47.745273 18149 ts_manager.cc:194] Registered new tserver with Master: 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117)
I20260807 18:24:47.746090 18149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:59201
I20260807 18:24:47.756968 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:24:47.763972 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.758028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.29.156.254:37923)
0807 18:24:47.758312 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.758318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.758622 (+ 304us) server_negotiation.cc:407] Connection header received
0807 18:24:47.758805 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.758808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.758952 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.759075 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:24:47.759500 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.760247 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.761804 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.762206 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.762302 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.762773 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:24:47.763282 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:24:47.763391 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.763588 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.763663 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 18:24:47.765012 18149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:24:47.773921 18149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:24:47.778887 18415 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:47.810904 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.787877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41546 (local address 127.29.156.194:43355)
0807 18:24:47.800380 (+ 12503us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.800395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.801537 (+ 1142us) server_negotiation.cc:407] Connection header received
0807 18:24:47.802015 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.802025 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.802323 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.802496 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:47.803396 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.804892 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.807813 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.809355 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.809959 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.810173 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.810316 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.810503 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12236,"thread_start_us":12523,"threads_started":1}
I20260807 18:24:47.810968 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.787298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43355 (local address 127.0.0.1:41546)
0807 18:24:47.801359 (+ 14061us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.801408 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.801721 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.802851 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.802856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.802890 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:47.803207 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.803215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.805120 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.805126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.807639 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.807653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.809630 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.809642 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.809853 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.810129 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.810276 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.810489 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13791,"mutex_wait_us":164,"thread_start_us":15454,"threads_started":1}
I20260807 18:24:47.814198 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.787331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:43671 (local address 127.0.0.1:38720)
0807 18:24:47.802490 (+ 15159us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.802509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.802647 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.805220 (+ 2573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.805225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.805269 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:47.805841 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.805907 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.807388 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.807394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.809589 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.809606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.813279 (+ 3673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.813286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.813478 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.813523 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.813612 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.813754 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15032,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":13579,"threads_started":1}
I20260807 18:24:47.815049 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.787685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.29.156.193:43671)
0807 18:24:47.804474 (+ 16789us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.804482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.804507 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:24:47.804589 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.804595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.804782 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.804944 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:47.806129 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.807233 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.809977 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.812684 (+ 2707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.812868 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.813036 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.814494 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.814691 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16606,"thread_start_us":16408,"threads_started":1}
I20260807 18:24:47.815979 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.788204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33117 (local address 127.0.0.1:50430)
0807 18:24:47.804050 (+ 15846us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.804069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.804202 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.805220 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.805225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.805274 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:47.807087 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.807116 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.809398 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.809405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.811654 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.811673 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.815083 (+ 3410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.815090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.815271 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.815303 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.815399 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.815588 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15697,"mutex_wait_us":31,"spinlock_wait_cycles":3328,"thread_start_us":15592,"threads_started":1}
I20260807 18:24:47.815778 18508 tablet_service.cc:1467] Processing CreateTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (DEFAULT_TABLE table=test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:47.816458 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.789017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.29.156.195:33117)
0807 18:24:47.803909 (+ 14892us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.803920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.804334 (+ 414us) server_negotiation.cc:407] Connection header received
0807 18:24:47.804442 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.804449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.804716 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.804926 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:47.807322 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.809018 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.811856 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.814548 (+ 2692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.814749 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.814923 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.815724 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.815921 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14668,"thread_start_us":14525,"threads_started":1}
I20260807 18:24:47.817235 18508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:47.818739 18323 tablet_service.cc:1467] Processing CreateTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (DEFAULT_TABLE table=test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:47.820065 18323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:47.826978 18690 tablet_service.cc:1467] Processing CreateTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (DEFAULT_TABLE table=test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:24:47.828444 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:24:47.832973 18800 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Bootstrap starting.
I20260807 18:24:47.835840 18800 tablet_bootstrap.cc:654] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:47.836480 18801 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Bootstrap starting.
I20260807 18:24:47.837260 18800 log.cc:824] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:47.840247 18801 tablet_bootstrap.cc:654] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:47.840627 18800 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: No bootstrap required, opened a new log
I20260807 18:24:47.840957 18800 ts_tablet_manager.cc:1397] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.005s
I20260807 18:24:47.841728 18801 log.cc:824] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:47.844879 18809 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Bootstrap starting.
I20260807 18:24:47.844964 18801 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: No bootstrap required, opened a new log
I20260807 18:24:47.845247 18801 ts_tablet_manager.cc:1397] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:24:47.848259 18809 tablet_bootstrap.cc:654] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Neither blocks nor log segments found. Creating new log.
I20260807 18:24:47.849330 18809 log.cc:824] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:47.851904 18800 raft_consensus.cc:348] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.852393 18800 raft_consensus.cc:374] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:47.852492 18800 raft_consensus.cc:729] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fcfaf016c0463bb78e3a010ed81def, State: Initialized, Role: FOLLOWER
I20260807 18:24:47.852794 18809 tablet_bootstrap.cc:492] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: No bootstrap required, opened a new log
I20260807 18:24:47.853087 18809 ts_tablet_manager.cc:1397] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 18:24:47.853225 18800 consensus_queue.cc:260] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.856405 18800 ts_tablet_manager.cc:1428] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 18:24:47.856153 18801 raft_consensus.cc:348] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.856637 18801 raft_consensus.cc:374] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:47.856745 18801 raft_consensus.cc:729] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483c56ebca334d34869ec51e70882f4c, State: Initialized, Role: FOLLOWER
I20260807 18:24:47.857362 18801 consensus_queue.cc:260] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.858582 18601 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:47.859227 18801 ts_tablet_manager.cc:1428] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 18:24:47.863253 18809 raft_consensus.cc:348] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.863760 18809 raft_consensus.cc:374] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:24:47.863886 18809 raft_consensus.cc:729] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2890f67badf9486bb3eca41b24c6316e, State: Initialized, Role: FOLLOWER
I20260807 18:24:47.864418 18809 consensus_queue.cc:260] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.865542 18774 heartbeater.cc:503] Master 127.29.156.254:37923 was elected leader, sending a full tablet report...
I20260807 18:24:47.866259 18809 ts_tablet_manager.cc:1428] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:24:47.940989 18814 raft_consensus.cc:482] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:24:47.941318 18814 raft_consensus.cc:504] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.943301 18814 leader_election.cc:288] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355), 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117)
I20260807 18:24:47.950667 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.943968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:41197 (local address 127.29.156.195:33117)
0807 18:24:47.944237 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.944245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.944663 (+ 418us) server_negotiation.cc:407] Connection header received
0807 18:24:47.944923 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.944927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.945021 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.945146 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:47.946096 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.946796 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.948890 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.949707 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.949885 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.949958 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.950262 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.950389 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:24:47.950685 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.943703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:60355 (local address 127.29.156.194:43355)
0807 18:24:47.943928 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:47.943936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:47.945011 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 18:24:47.945174 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:47.945179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:47.945249 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:47.945342 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:24:47.946068 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.946789 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.948848 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.949626 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.949748 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.949845 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:47.950295 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 18:24:47.950422 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:24:47.950992 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.943702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33117 (local address 127.29.156.193:41197)
0807 18:24:47.944526 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.944541 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.944761 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.945476 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.945482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.945502 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:47.945909 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.945918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.946981 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.946985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.948698 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.948711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.949952 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.949960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.950077 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.950129 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.950202 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.950364 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":61,"thread_start_us":436,"threads_started":1}
I20260807 18:24:47.951025 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:47.943755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:43355 (local address 127.29.156.193:60355)
0807 18:24:47.944886 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:47.944902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:24:47.945032 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:24:47.945397 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:24:47.945401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:24:47.945426 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:24:47.945886 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.945899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.946948 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.946956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.948686 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:24:47.948698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:24:47.949877 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:24:47.949884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:24:47.950060 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:47.950135 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 18:24:47.950243 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:24:47.950369 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":60,"spinlock_wait_cycles":17792,"thread_start_us":738,"threads_started":1}
I20260807 18:24:47.952191 18537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e5ff1cf8994d08a3643cf9bcb1a10b" candidate_uuid: "56fcfaf016c0463bb78e3a010ed81def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483c56ebca334d34869ec51e70882f4c" is_pre_election: true
I20260807 18:24:47.952174 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e5ff1cf8994d08a3643cf9bcb1a10b" candidate_uuid: "56fcfaf016c0463bb78e3a010ed81def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2890f67badf9486bb3eca41b24c6316e" is_pre_election: true
I20260807 18:24:47.952762 18537 raft_consensus.cc:2454] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56fcfaf016c0463bb78e3a010ed81def in term 0.
I20260807 18:24:47.952762 18717 raft_consensus.cc:2454] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56fcfaf016c0463bb78e3a010ed81def in term 0.
I20260807 18:24:47.953513 18256 leader_election.cc:302] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483c56ebca334d34869ec51e70882f4c, 56fcfaf016c0463bb78e3a010ed81def; no voters:
I20260807 18:24:47.953927 18814 raft_consensus.cc:2781] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:24:47.954093 18814 raft_consensus.cc:482] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:24:47.954238 18814 raft_consensus.cc:3037] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:47.956743 18814 raft_consensus.cc:504] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
I20260807 18:24:47.957388 18814 leader_election.cc:288] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [CANDIDATE]: Term 1 election: Requested vote from peers 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355), 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117)
I20260807 18:24:47.957921 18537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e5ff1cf8994d08a3643cf9bcb1a10b" candidate_uuid: "56fcfaf016c0463bb78e3a010ed81def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483c56ebca334d34869ec51e70882f4c"
I20260807 18:24:47.958137 18537 raft_consensus.cc:3037] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:47.958150 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e5ff1cf8994d08a3643cf9bcb1a10b" candidate_uuid: "56fcfaf016c0463bb78e3a010ed81def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2890f67badf9486bb3eca41b24c6316e"
I20260807 18:24:47.958416 18717 raft_consensus.cc:3037] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:24:47.960711 18537 raft_consensus.cc:2454] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56fcfaf016c0463bb78e3a010ed81def in term 1.
I20260807 18:24:47.961165 18256 leader_election.cc:302] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483c56ebca334d34869ec51e70882f4c, 56fcfaf016c0463bb78e3a010ed81def; no voters:
I20260807 18:24:47.961151 18717 raft_consensus.cc:2454] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56fcfaf016c0463bb78e3a010ed81def in term 1.
I20260807 18:24:47.961536 18814 raft_consensus.cc:2781] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:24:47.962051 18814 raft_consensus.cc:686] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 1 LEADER]: Becoming Leader. State: Replica: 56fcfaf016c0463bb78e3a010ed81def, State: Running, Role: LEADER
I20260807 18:24:47.962584 18814 consensus_queue.cc:237] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } }
W20260807 18:24:47.968214 18777 tablet.cc:2369] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:47.967947 18149 catalog_manager.cc:5495] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def reported cstate change: term changed from 0 to 1, leader changed from <none> to 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193). New cstate: current_term: 1 leader_uuid: "56fcfaf016c0463bb78e3a010ed81def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fcfaf016c0463bb78e3a010ed81def" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 } health_report { overall_health: UNKNOWN } } }
W20260807 18:24:47.981921 18602 tablet.cc:2369] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:24:47.993283 18416 tablet.cc:2369] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:24:48.007691 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:48.001983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.29.156.193:43671)
0807 18:24:48.002216 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:48.002222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:48.002243 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:24:48.002488 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:48.002492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:48.002575 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:48.002743 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:24:48.003145 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.003953 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.005368 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.005789 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.005928 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:48.006532 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:24:48.007055 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:24:48.007218 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:48.007280 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:24:48.007366 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:24:48.016573 18717 raft_consensus.cc:1261] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Refusing update from remote peer 56fcfaf016c0463bb78e3a010ed81def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:48.016564 18537 raft_consensus.cc:1261] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Refusing update from remote peer 56fcfaf016c0463bb78e3a010ed81def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:24:48.017765 18822 consensus_queue.cc:1035] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [LEADER]: Connected to new peer: Peer: permanent_uuid: "483c56ebca334d34869ec51e70882f4c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:48.018280 18814 consensus_queue.cc:1035] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [LEADER]: Connected to new peer: Peer: permanent_uuid: "2890f67badf9486bb3eca41b24c6316e" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:24:48.049919 18149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:48.050230 18149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:24:48.054229 18149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Sending DeleteTablet for 3 replicas of tablet b2e5ff1cf8994d08a3643cf9bcb1a10b
I20260807 18:24:48.055572 18323 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:48 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:38720
I20260807 18:24:48.055760 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:48 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:50430
I20260807 18:24:48.055773 18508 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e5ff1cf8994d08a3643cf9bcb1a10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:24:48 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:41546
I20260807 18:24:48.056478 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 18198
I20260807 18:24:48.058419 18840 tablet_replica.cc:330] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: stopping tablet replica
I20260807 18:24:48.059073 18840 raft_consensus.cc:2229] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:24:48.060009 18840 raft_consensus.cc:2258] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:48.061000 18842 tablet_replica.cc:330] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: stopping tablet replica
I20260807 18:24:48.061486 18842 raft_consensus.cc:2229] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:48.062054 18842 raft_consensus.cc:2258] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:48.063721 18840 ts_tablet_manager.cc:1905] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:48.068341 18842 ts_tablet_manager.cc:1905] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:48.073712 18840 ts_tablet_manager.cc:1918] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:48.074110 18840 log.cc:1196] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/b2e5ff1cf8994d08a3643cf9bcb1a10b
I20260807 18:24:48.075028 18840 ts_tablet_manager.cc:1939] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 56fcfaf016c0463bb78e3a010ed81def: Deleting consensus metadata
I20260807 18:24:48.077417 18125 catalog_manager.cc:4841] TS 56fcfaf016c0463bb78e3a010ed81def (127.29.156.193:43671): tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (table test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]) successfully deleted
I20260807 18:24:48.080088 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:48.057518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.29.156.193:43671)
0807 18:24:48.058250 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:48.058257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:48.058275 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:24:48.058332 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:48.058337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:48.058416 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:48.058516 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:48.060674 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.061899 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.064000 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.064786 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.064919 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:48.070018 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:48.070034 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:48.070051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:48.070127 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:48.074695 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.075758 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.075767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.075771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.075954 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.076505 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.076511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.076514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.076987 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:48.078008 (+ 1021us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:48.079268 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0807 18:24:48.079519 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600}
I20260807 18:24:48.080842 18406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:48.082266 18842 ts_tablet_manager.cc:1918] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:48.082489 18842 log.cc:1196] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/b2e5ff1cf8994d08a3643cf9bcb1a10b
I20260807 18:24:48.082716 18841 tablet_replica.cc:330] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: stopping tablet replica
I20260807 18:24:48.083189 18842 ts_tablet_manager.cc:1939] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 483c56ebca334d34869ec51e70882f4c: Deleting consensus metadata
I20260807 18:24:48.083736 18841 raft_consensus.cc:2229] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:24:48.084786 18841 raft_consensus.cc:2258] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:24:48.085371 18122 catalog_manager.cc:4841] TS 483c56ebca334d34869ec51e70882f4c (127.29.156.194:43355): tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (table test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]) successfully deleted
I20260807 18:24:48.087056 18841 ts_tablet_manager.cc:1905] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:24:48.095580 18841 ts_tablet_manager.cc:1918] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:24:48.095779 18841 log.cc:1196] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/b2e5ff1cf8994d08a3643cf9bcb1a10b
I20260807 18:24:48.096474 18841 ts_tablet_manager.cc:1939] T b2e5ff1cf8994d08a3643cf9bcb1a10b P 2890f67badf9486bb3eca41b24c6316e: Deleting consensus metadata
I20260807 18:24:48.098758 18124 catalog_manager.cc:4841] TS 2890f67badf9486bb3eca41b24c6316e (127.29.156.195:33117): tablet b2e5ff1cf8994d08a3643cf9bcb1a10b (table test-table [id=1ad5bfb5b0374a5b96d45b52a384c573]) successfully deleted
I20260807 18:24:48.188417 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 18198
I20260807 18:24:48.215847 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 18421
I20260807 18:24:48.230554 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:48.216705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.29.156.194:43355)
0807 18:24:48.216890 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:48.216895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:48.216913 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:24:48.216975 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:48.216979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:48.217054 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:48.217148 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:48.218627 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.219428 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.220846 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.221289 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.221452 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:48.224405 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:48.224417 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:48.224430 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:48.224481 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:48.227660 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.228362 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.228378 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.228380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.228510 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.228903 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.228908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.228911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.229308 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:48.229628 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:48.229940 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 18:24:48.230163 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:24:48.231432 18595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:48.325332 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 18421
I20260807 18:24:48.352094 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 18604
I20260807 18:24:48.367170 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:48.353015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.29.156.195:33117)
0807 18:24:48.353235 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:24:48.353243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:24:48.353327 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:24:48.353598 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:24:48.353603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:24:48.353673 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:24:48.353790 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:24:48.355311 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.356210 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.357647 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:24:48.358102 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:24:48.358251 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:24:48.361328 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:24:48.361343 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:24:48.361359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:24:48.361398 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:24:48.364231 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.364929 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.364937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.364939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.365108 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:24:48.365493 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:24:48.365512 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:24:48.365515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:24:48.365881 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:24:48.366194 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:24:48.366572 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 18:24:48.366834 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:24:48.367986 18769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:48.457468 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 18604
I20260807 18:24:48.483037 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 18096
I20260807 18:24:48.484378 18190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:24:48.603528 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 18096
I20260807 18:24:48.637791 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--webserver_interface=127.29.156.254
--webserver_port=42537
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:24:48.991690 18852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:48.992018 18852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:48.992219 18852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:49.000012 18852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:24:49.000151 18852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:49.000257 18852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:49.000376 18852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:24:49.000444 18852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:24:49.010777 18852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=42537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:49.012348 18852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:49.013731 18852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:49.026592 18866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:49.026760 18861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:49.027271 18859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:49.027678 18852 server_base.cc:1034] running on GCE node
I20260807 18:24:49.028595 18852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:49.030226 18852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:49.031473 18852 hybrid_clock.cc:648] HybridClock initialized: now 1786127089031403 us; error 68 us; skew 500 ppm
I20260807 18:24:49.031898 18852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:49 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127089, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:49.035226 18852 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:24:49.037034 18852 webserver.cc:460] Webserver started at http://127.29.156.254:42537/ using document root <none> and password file <none>
I20260807 18:24:49.037875 18852 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:49.038033 18852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:49.043857 18852 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:24:49.049211 18873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:49.050729 18852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:24:49.050925 18852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "7abe45c875d14f52862bd33b14ed914d"
format_stamp: "Formatted at 2026-08-07 18:24:42 on dist-test-slave-sb2z"
server_key: "765cbf9f196e6d03bec6f0b3eaf39b2b"
server_key_iv: "433fe1402489fb2ec0f74ff52b191633"
server_key_version: "encryptionkey@0"
I20260807 18:24:49.052034 18852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:49.074137 18852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:49.078928 18852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:49.079272 18852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:49.100390 18852 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37923
I20260807 18:24:49.100414 18945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37923 every 8 connection(s)
I20260807 18:24:49.102207 18852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:24:49.109591 18946 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:24:49.110529 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18852
I20260807 18:24:49.111572 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:43671
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=34429
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:49.114797 18946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap starting.
I20260807 18:24:49.139101 18946 log.cc:824] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:24:49.156688 18946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:24:49.157328 18946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d: Bootstrap complete.
I20260807 18:24:49.167052 18946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:49.168048 18946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Initialized, Role: FOLLOWER
I20260807 18:24:49.168684 18946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:49.168926 18946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:24:49.169045 18946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:24:49.169253 18946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:24:49.173439 18946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:49.173988 18946 leader_election.cc:302] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7abe45c875d14f52862bd33b14ed914d; no voters:
I20260807 18:24:49.174711 18946 leader_election.cc:288] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:24:49.174863 18950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:24:49.176151 18950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [term 3 LEADER]: Becoming Leader. State: Replica: 7abe45c875d14f52862bd33b14ed914d, State: Running, Role: LEADER
I20260807 18:24:49.176837 18950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } }
I20260807 18:24:49.177208 18946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:24:49.179659 18954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7abe45c875d14f52862bd33b14ed914d. Latest consensus state: current_term: 3 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:49.180091 18954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:49.180730 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7abe45c875d14f52862bd33b14ed914d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abe45c875d14f52862bd33b14ed914d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37923 } } }
I20260807 18:24:49.181068 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abe45c875d14f52862bd33b14ed914d [sys.catalog]: This master's current role is: LEADER
I20260807 18:24:49.187866 18961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:24:49.196720 18961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:24:49.198516 18961 catalog_manager.cc:1259] Loaded cluster ID: 71ffb18cdb7040b7b6666bcf3ae39cdc
I20260807 18:24:49.198685 18961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:24:49.201020 18961 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4cf56999ad google::LogMessage::Fail() at ??:0
@ 0x7f4cf569c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f4cf56994de google::LogMessage::Flush() at ??:0
@ 0x7f4cf569aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4d0b0cc512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f4d0b0c9976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4d0b151610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4d0b151439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4cf82e5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4cf82e891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4cf82e8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4cf82c3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4cfb59d6db start_thread at ??:0
@ 0x7f4cf453361f clone at ??:0
W20260807 18:24:49.512547 18948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:49.512861 18948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:49.513042 18948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:49.520531 18948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:49.520684 18948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:49.520862 18948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:24:49.530833 18948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=34429
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:49.532192 18948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:49.533569 18948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:49.547573 19001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:49.547574 19006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:49.548300 19000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:49.548336 18948 server_base.cc:1034] running on GCE node
I20260807 18:24:49.549042 18948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:49.550087 18948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:49.551308 18948 hybrid_clock.cc:648] HybridClock initialized: now 1786127089551234 us; error 63 us; skew 500 ppm
I20260807 18:24:49.551724 18948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:49 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127089, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:49.555281 18948 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:24:49.557065 18948 webserver.cc:460] Webserver started at http://127.29.156.193:34429/ using document root <none> and password file <none>
I20260807 18:24:49.557798 18948 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:49.557957 18948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:49.563634 18948 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:24:49.566382 19016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:49.567855 18948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:24:49.568040 18948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "56fcfaf016c0463bb78e3a010ed81def"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "fee6cba6c49e19ea39aa8c85041d868f"
server_key_iv: "8dcaf31364c2ed9bf9f9e507c741f760"
server_key_version: "encryptionkey@0"
I20260807 18:24:49.569027 18948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:49.603835 18948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:49.608384 18948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:49.608700 18948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:49.609781 18948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:49.611220 18948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:49.611356 18948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:49.611500 18948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:49.611621 18948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:24:49.648003 18948 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:43671
I20260807 18:24:49.648062 19187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:43671 every 8 connection(s)
I20260807 18:24:49.649638 18948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:24:49.653784 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 18948
I20260807 18:24:49.654661 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:43355
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=44139
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:49.655306 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:49.652793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:40357)
0807 18:24:49.653768 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:49.654084 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":384,"threads_started":1}
W20260807 18:24:49.656141 19189 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:37923 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:37923: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
I20260807 18:24:49.657179 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:49.656714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:52517)
0807 18:24:49.656914 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:49.657014 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:24:49.658202 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:49.657757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:41789)
0807 18:24:49.657928 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:49.658028 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:24:49.658492 19189 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:24:50.009320 19195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:50.009624 19195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:50.009800 19195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:50.017107 19195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:50.017220 19195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:50.017403 19195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:24:50.028107 19195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:43355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=44139
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:50.029572 19195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:50.031081 19195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:50.046144 19203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:50.046507 19206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:50.046341 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:50.047020 19195 server_base.cc:1034] running on GCE node
I20260807 18:24:50.047879 19195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:50.048836 19195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:50.050069 19195 hybrid_clock.cc:648] HybridClock initialized: now 1786127090050009 us; error 41 us; skew 500 ppm
I20260807 18:24:50.050422 19195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:50 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127090, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:50.053839 19195 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:24:50.055368 19195 webserver.cc:460] Webserver started at http://127.29.156.194:44139/ using document root <none> and password file <none>
I20260807 18:24:50.056135 19195 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:50.056278 19195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:50.061805 19195 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:24:50.065138 19215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.066880 19195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:24:50.067042 19195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "483c56ebca334d34869ec51e70882f4c"
format_stamp: "Formatted at 2026-08-07 18:24:43 on dist-test-slave-sb2z"
server_key: "2fbb64c365741f7336ca451c4fdcea3d"
server_key_iv: "918da2ad5bb13155b124bb280ffa1876"
server_key_version: "encryptionkey@0"
I20260807 18:24:50.068199 19195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:50.087087 19195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:50.091919 19195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:50.092186 19195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:50.093230 19195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:50.094682 19195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:50.094780 19195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.094866 19195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:50.094931 19195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.133780 19195 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:43355
I20260807 18:24:50.133867 19392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:43355 every 8 connection(s)
I20260807 18:24:50.135439 19195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:24:50.137588 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 19195
I20260807 18:24:50.138525 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:33117
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:37923
--webserver_port=45285
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:24:50.141418 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.139243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:60893)
0807 18:24:50.140174 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.140481 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":395,"threads_started":1}
W20260807 18:24:50.142261 19393 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:37923 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:37923: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
I20260807 18:24:50.143260 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.142904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:39379)
0807 18:24:50.143067 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.143146 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:24:50.144393 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.143900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:52231)
0807 18:24:50.144097 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.144216 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:24:50.144683 19393 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:24:50.491412 19396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:24:50.491725 19396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:24:50.491888 19396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:24:50.499178 19396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:24:50.499294 19396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:24:50.499547 19396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:24:50.509388 19396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:33117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=45285
--tserver_master_addrs=127.29.156.254:37923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:24:50.510812 19396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:24:50.512260 19396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:24:50.530233 19408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:50.530284 19406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:24:50.530331 19405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:24:50.531386 19396 server_base.cc:1034] running on GCE node
I20260807 18:24:50.532116 19396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:24:50.533169 19396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:24:50.534409 19396 hybrid_clock.cc:648] HybridClock initialized: now 1786127090534376 us; error 59 us; skew 500 ppm
I20260807 18:24:50.534749 19396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:24:50 dist-test-slave-sb2z krb5kdc[17298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127090, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:24:50.541668 19396 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:24:50.543380 19396 webserver.cc:460] Webserver started at http://127.29.156.195:45285/ using document root <none> and password file <none>
I20260807 18:24:50.544198 19396 fs_manager.cc:359] Metadata directory not provided
I20260807 18:24:50.544361 19396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:24:50.549924 19396 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:24:50.553059 19418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.554455 19396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:24:50.554651 19396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "2890f67badf9486bb3eca41b24c6316e"
format_stamp: "Formatted at 2026-08-07 18:24:44 on dist-test-slave-sb2z"
server_key: "238bb83b53ff0612447a6bf8cb70a01e"
server_key_iv: "46713f5ea09d8a2fcb379482dfc4768a"
server_key_version: "encryptionkey@0"
I20260807 18:24:50.555706 19396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:24:50.572347 19396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:24:50.576804 19396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:24:50.577098 19396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:24:50.578087 19396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:24:50.579447 19396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:24:50.579605 19396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.579738 19396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:24:50.579834 19396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:24:50.615746 19396 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33117
I20260807 18:24:50.615833 19594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33117 every 8 connection(s)
I20260807 18:24:50.617260 19396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:24:50.622196 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.620335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:56045)
0807 18:24:50.621077 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.621295 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":272,"threads_started":1}
I20260807 18:24:50.622764 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 19396
W20260807 18:24:50.622829 19597 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:37923 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:37923: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
I20260807 18:24:50.623890 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.623450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:46627)
0807 18:24:50.623651 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.623746 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:24:50.624811 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.624403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:37035)
0807 18:24:50.624560 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.624649 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:24:50.625130 19597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:24:50.660105 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:50.659000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:45569)
0807 18:24:50.659619 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:50.659748 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":320,"threads_started":1}
I20260807 18:24:51.146502 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:51.145339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:39465)
0807 18:24:51.145915 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:51.146145 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":295,"threads_started":1}
I20260807 18:24:51.626685 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:51.625726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:58143)
0807 18:24:51.626260 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:51.626403 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":266,"threads_started":1}
I20260807 18:24:51.661938 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:51.661183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:40207)
0807 18:24:51.661680 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:51.661766 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260807 18:24:52.148244 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:52.147426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:56267)
0807 18:24:52.147977 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:52.148067 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":293,"threads_started":1}
I20260807 18:24:52.628635 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:52.627767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:40301)
0807 18:24:52.628352 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:52.628462 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":332,"threads_started":1}
I20260807 18:24:52.663877 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:52.662947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:54843)
0807 18:24:52.663440 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:52.663679 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":282,"threads_started":1}
I20260807 18:24:53.149984 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:53.149276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:56567)
0807 18:24:53.149730 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:53.149811 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260807 18:24:53.630545 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:53.629611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:58195)
0807 18:24:53.630160 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:53.630311 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":254,"threads_started":1}
I20260807 18:24:53.665616 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:53.664863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:48063)
0807 18:24:53.665343 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:53.665431 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":241,"threads_started":1}
I20260807 18:24:54.152005 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:54.151016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:39027)
0807 18:24:54.151679 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:54.151785 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":327,"threads_started":1}
I20260807 18:24:54.632536 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:54.631713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:35529)
0807 18:24:54.632243 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:54.632335 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":279,"threads_started":1}
I20260807 18:24:54.667312 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:54.666588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:51159)
0807 18:24:54.667057 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:54.667141 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":242,"threads_started":1}
I20260807 18:24:55.153786 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:55.152990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:52277)
0807 18:24:55.153497 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:55.153586 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":252,"threads_started":1}
I20260807 18:24:55.634627 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:55.633656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:54551)
0807 18:24:55.634267 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:55.634369 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":326,"threads_started":1}
I20260807 18:24:55.669245 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:55.668396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:57855)
0807 18:24:55.668958 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:55.669072 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":332,"threads_started":1}
I20260807 18:24:56.155900 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:56.154867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:48543)
0807 18:24:56.155504 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:56.155694 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":364,"threads_started":1}
I20260807 18:24:56.636727 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:56.635810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:51639)
0807 18:24:56.636427 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:56.636540 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":304,"threads_started":1}
I20260807 18:24:56.671365 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:56.670418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:46423)
0807 18:24:56.671033 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:56.671138 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":243,"threads_started":1}
I20260807 18:24:57.158245 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:57.157115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:59589)
0807 18:24:57.157817 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:57.157967 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":238,"threads_started":1}
I20260807 18:24:57.638798 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:57.637897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:56231)
0807 18:24:57.638483 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:57.638572 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":289,"threads_started":1}
I20260807 18:24:57.673233 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:57.672438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:36213)
0807 18:24:57.672953 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:57.673063 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":314,"threads_started":1}
I20260807 18:24:58.160465 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:58.159489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:59173)
0807 18:24:58.160191 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:58.160277 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":309,"threads_started":1}
I20260807 18:24:58.641960 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:58.640636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:51659)
0807 18:24:58.641646 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:58.641739 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":758,"threads_started":1}
I20260807 18:24:58.675083 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:58.674208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:40051)
0807 18:24:58.674763 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:58.674884 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":266,"threads_started":1}
I20260807 18:24:59.162262 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:59.161455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:42405)
0807 18:24:59.161976 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:59.162076 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":263,"threads_started":1}
I20260807 18:24:59.643904 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:59.642984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:41495)
0807 18:24:59.643596 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:59.643712 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":254,"threads_started":1}
I20260807 18:24:59.676856 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:24:59.676123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:42961)
0807 18:24:59.676610 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 18:24:59.676694 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260807 18:25:00.164186 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:00.163345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:50757)
0807 18:25:00.163906 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:00.164000 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":342,"threads_started":1}
I20260807 18:25:00.646104 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:00.645052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:44861)
0807 18:25:00.645732 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:00.645857 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":352,"threads_started":1}
I20260807 18:25:00.678849 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:00.677870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:33453)
0807 18:25:00.678496 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:00.678621 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":254,"threads_started":1}
I20260807 18:25:01.165979 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:01.165223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:40279)
0807 18:25:01.165685 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:01.165782 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260807 18:25:01.648061 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:01.647177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:58911)
0807 18:25:01.647725 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:01.647822 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":240,"threads_started":1}
I20260807 18:25:01.680665 19650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:01.679913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:34731)
0807 18:25:01.680417 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:01.680502 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":277,"threads_started":1}
I20260807 18:25:02.167779 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:02.166964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:36555)
0807 18:25:02.167452 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:02.167615 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":265,"threads_started":1}
I20260807 18:25:02.650097 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:02.649236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:42771)
0807 18:25:02.649826 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:02.649918 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":331,"threads_started":1}
I20260807 18:25:02.682612 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:02.681719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:38763)
0807 18:25:02.682306 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:02.682415 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":313,"threads_started":1}
I20260807 18:25:03.169571 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:03.168754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:51583)
0807 18:25:03.169248 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:03.169343 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":237,"threads_started":1}
I20260807 18:25:03.652181 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:03.651189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:41839)
0807 18:25:03.651809 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:03.651935 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":263,"threads_started":1}
I20260807 18:25:03.684547 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:03.683749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:43641)
0807 18:25:03.684249 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:03.684343 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":230,"threads_started":1}
I20260807 18:25:04.171450 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:04.170594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:33945)
0807 18:25:04.171137 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:04.171253 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":257,"threads_started":1}
I20260807 18:25:04.654203 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:04.653459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:33069)
0807 18:25:04.653914 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:04.654009 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260807 18:25:04.686610 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:04.685773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:50815)
0807 18:25:04.686280 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:04.686386 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":256,"threads_started":1}
I20260807 18:25:05.174012 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.172902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.194:60343)
0807 18:25:05.173544 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:05.173732 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":327,"threads_started":1}
I20260807 18:25:05.628253 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 18948
I20260807 18:25:05.647770 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.629474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.29.156.193:43671)
0807 18:25:05.630125 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:05.630140 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:05.630181 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:25:05.632090 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:05.632101 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:05.632568 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:05.632891 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:05.634458 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.635860 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.637214 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.637912 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.638052 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:05.641151 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:05.641159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:05.641176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:05.641217 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:05.644318 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.645020 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.645028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.645034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.645216 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.645598 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.645602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.645604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.646170 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:05.646408 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:05.646868 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 18:25:05.647364 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":315,"threads_started":1}
I20260807 18:25:05.649360 19181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:05.656440 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.655589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.195:33827)
0807 18:25:05.656139 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:05.656257 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":268,"threads_started":1}
I20260807 18:25:05.738144 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.736919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37923 (local address 127.29.156.193:58227)
0807 18:25:05.737551 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:05.737854 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":295,"threads_started":1}
I20260807 18:25:05.742648 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 18948
I20260807 18:25:05.772647 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 19195
I20260807 18:25:05.792194 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.773792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.29.156.194:43355)
0807 18:25:05.774427 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:05.774442 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:05.774482 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:25:05.776208 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:05.776219 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:05.776667 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:05.776967 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:05.778473 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.779792 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.781114 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.781828 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.781976 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:05.785079 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:05.785087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:05.785104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:05.785148 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:05.788343 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.789109 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.789117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.789124 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.789325 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.789780 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.789785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.789787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.790488 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:05.790785 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:05.791331 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 18:25:05.791859 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":313,"threads_started":1}
I20260807 18:25:05.793725 19386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:05.882009 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 19195
I20260807 18:25:05.910142 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 19396
I20260807 18:25:05.929786 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:05.911386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.29.156.195:33117)
0807 18:25:05.912192 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:05.912206 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:05.912249 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:25:05.913877 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:05.913886 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:05.914325 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:05.914714 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:05.916336 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.917622 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.918954 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:05.919724 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:05.919947 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:05.923199 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:05.923208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:05.923224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:05.923267 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:05.926233 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.926936 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.926944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.926951 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.927150 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:05.927585 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:05.927592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:05.927595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:05.928177 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:05.928538 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:05.929013 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 18:25:05.929471 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":401,"threads_started":1}
I20260807 18:25:05.931349 19590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:06.018513 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 19396
2026-08-07T18:25:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25685 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:25:06 dist-test-slave-sb2z krb5kdc[19682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:25:06 dist-test-slave-sb2z krb5kdc[19682](info): set up 2 sockets
Aug 07 18:25:06 dist-test-slave-sb2z krb5kdc[19682](info): commencing operation
krb5kdc: starting...
W20260807 18:25:07.343070 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:25:07 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:25:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:25:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:25:07.676669 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:25:08.031584 19702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:08.031906 19702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:08.032069 19702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:08.039636 19702 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:25:08.039752 19702 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:25:08.039814 19702 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:25:08.039891 19702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:25:08.039981 19702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:08.040082 19702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:08.040170 19702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:25:08.040251 19702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:25:08.050483 19702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:08.051854 19702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:08.053246 19702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:08.066251 19713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:08.066309 19715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:08.066309 19709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:08.066937 19702 server_base.cc:1034] running on GCE node
I20260807 18:25:08.067924 19702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:08.069399 19702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:08.070708 19702 hybrid_clock.cc:648] HybridClock initialized: now 1786127108070602 us; error 57 us; skew 500 ppm
I20260807 18:25:08.071158 19702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:08 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127108, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:08.079749 19702 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:25:08.081624 19702 webserver.cc:460] Webserver started at http://127.29.156.254:46563/ using document root <none> and password file <none>
I20260807 18:25:08.082440 19702 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:08.082593 19702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:08.082963 19702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:25:08.086211 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "cdd7031236594757b48dcb1fa09a9442"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "63588b8c50ddde6abf1edf1239eba542"
server_key_iv: "243fd0af60eaafadebb73f415285ca12"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.087066 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "cdd7031236594757b48dcb1fa09a9442"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "63588b8c50ddde6abf1edf1239eba542"
server_key_iv: "243fd0af60eaafadebb73f415285ca12"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.092756 19702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 18:25:08.096690 19726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:08.098371 19702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:25:08.098702 19702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "cdd7031236594757b48dcb1fa09a9442"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "63588b8c50ddde6abf1edf1239eba542"
server_key_iv: "243fd0af60eaafadebb73f415285ca12"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.098965 19702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:08.127413 19702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:08.132361 19702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:08.132692 19702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:08.152990 19702 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44025
I20260807 18:25:08.153009 19799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44025 every 8 connection(s)
I20260807 18:25:08.154501 19702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:25:08.158684 19802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:08.160096 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 19702
I20260807 18:25:08.160351 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:25:08.161300 30323 external_mini_cluster.cc:1428] Setting key 4972a1a67af7f4409534f53813c18f68
I20260807 18:25:08.173738 19802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap starting.
I20260807 18:25:08.178284 19802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Neither blocks nor log segments found. Creating new log.
Aug 07 18:25:08 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:08.179813 19802 log.cc:824] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:08.183492 19802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: No bootstrap required, opened a new log
I20260807 18:25:08.189522 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:08.163694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.29.156.254:44025)
0807 18:25:08.164630 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:08.164642 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:08.164685 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:25:08.166363 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:08.166389 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:08.166814 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:08.167150 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:08.169342 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.170923 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.173092 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.174070 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.174256 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:08.179885 (+ 5629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:08.179895 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:08.179912 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:08.180015 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:08.184197 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:08.185318 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:08.185325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:08.185332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:08.185566 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:08.186252 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:08.186256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:08.186258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:08.186912 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:08.187184 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:08.187784 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:25:08.188374 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":431,"threads_started":1}
I20260807 18:25:08.192971 19802 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:08.193389 19802 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:08.193504 19802 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Initialized, Role: FOLLOWER
I20260807 18:25:08.194126 19802 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:08.194384 19802 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:25:08.194553 19802 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:25:08.194736 19802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:08.197103 19802 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:08.197641 19802 leader_election.cc:302] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdd7031236594757b48dcb1fa09a9442; no voters:
I20260807 18:25:08.198338 19802 leader_election.cc:288] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:25:08.198539 19809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:25:08.199936 19809 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 LEADER]: Becoming Leader. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Running, Role: LEADER
I20260807 18:25:08.200819 19802 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:25:08.200659 19809 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:08.203719 19811 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd7031236594757b48dcb1fa09a9442. Latest consensus state: current_term: 1 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:08.204087 19811 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:08.204748 19810 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:08.205082 19810 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:08.207216 19817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:25:08.212966 19817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:25:08.222206 19817 catalog_manager.cc:1347] Generated new cluster ID: 775ac5c0b9fd444f9b8a2cbc64907e7d
I20260807 18:25:08.222344 19817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:25:08.238996 19817 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:25:08.240454 19817 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:25:08.252686 19817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Generated new TSK 0
I20260807 18:25:08.253579 19817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:25:08.340291 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:25:08.703734 19838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:08.704031 19838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:08.704223 19838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:08.711848 19838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:08.711963 19838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:08.712147 19838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:25:08.722393 19838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:08.723946 19838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:08.725343 19838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:08.740435 19847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:08.740449 19848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:08.740440 19853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:08.742095 19838 server_base.cc:1034] running on GCE node
I20260807 18:25:08.742733 19838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:08.743772 19838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:08.744988 19838 hybrid_clock.cc:648] HybridClock initialized: now 1786127108744927 us; error 52 us; skew 500 ppm
I20260807 18:25:08.745359 19838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:08 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127108, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:08.748845 19838 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:25:08.750401 19838 webserver.cc:460] Webserver started at http://127.29.156.193:43795/ using document root <none> and password file <none>
I20260807 18:25:08.751183 19838 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:08.751322 19838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:08.751746 19838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:25:08.755043 19838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "95f5894b2402434ead123b469c69a85e"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "bcdad0ed977377cf1062bcee3b2b9266"
server_key_iv: "8c4e6032242a3085146f8511d5dc898e"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.755930 19838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "95f5894b2402434ead123b469c69a85e"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "bcdad0ed977377cf1062bcee3b2b9266"
server_key_iv: "8c4e6032242a3085146f8511d5dc898e"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.761335 19838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:25:08.765189 19866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:08.766675 19838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:25:08.766860 19838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "95f5894b2402434ead123b469c69a85e"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "bcdad0ed977377cf1062bcee3b2b9266"
server_key_iv: "8c4e6032242a3085146f8511d5dc898e"
server_key_version: "encryptionkey@0"
I20260807 18:25:08.767042 19838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:08.779296 19838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:08.783888 19838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:08.784211 19838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:08.785300 19838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:08.786751 19838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:08.786826 19838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:08.786957 19838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:08.787026 19838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:08.824613 19838 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38015
I20260807 18:25:08.824661 20033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38015 every 8 connection(s)
I20260807 18:25:08.826001 19838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:25:08.835417 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 19838
I20260807 18:25:08.835682 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:25:08.836334 30323 external_mini_cluster.cc:1428] Setting key 96f0fac7bd595de53a4896c41101b84c
Aug 07 18:25:08 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127108, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:08.847906 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:08.828940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:49227 (local address 127.29.156.254:44025)
0807 18:25:08.829685 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:08.829692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:08.830659 (+ 967us) server_negotiation.cc:407] Connection header received
0807 18:25:08.832658 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:08.832663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:08.832734 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:08.832844 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:08.834333 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.835078 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.836882 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.837314 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.837458 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:08.841891 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:08.841896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:08.841900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:08.841937 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:08.844626 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:08.845436 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:08.845441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:08.845443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:08.845571 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:08.846056 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:08.846060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:08.846062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:08.846327 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:08.846503 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:08.847245 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 18:25:08.847490 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":474,"threads_started":1}
I20260807 18:25:08.848922 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:08.829445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:49227)
0807 18:25:08.830458 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:08.830505 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:08.832381 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:08.833057 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:08.833066 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:08.833618 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:08.834178 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:08.834195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.835265 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.835269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:08.836728 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:08.836746 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:08.837574 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:08.837583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:08.837837 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:08.838558 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:08.838581 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:08.841631 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:08.844793 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:08.844802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:08.844821 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:08.845289 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:08.845690 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:08.845695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:08.845699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:08.845912 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:08.846462 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:08.846471 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:08.846847 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 18:25:08.847128 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 18:25:08.847756 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":506,"threads_started":1}
I20260807 18:25:08.850770 20035 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:08.851127 20035 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:08.851938 20035 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:08.854030 19754 ts_manager.cc:194] Registered new tserver with Master: 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015)
I20260807 18:25:08.855513 19754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:49227
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:25:08.912096 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:25:09.266597 20046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:09.266891 20046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:09.266989 20046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:09.274564 20046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:09.274679 20046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:09.274796 20046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:25:09.284781 20046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:09.286128 20046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:09.287444 20046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:09.301241 20057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:09.301405 20051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:09.301343 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:09.302868 20046 server_base.cc:1034] running on GCE node
I20260807 18:25:09.303741 20046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:09.304791 20046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:09.306054 20046 hybrid_clock.cc:648] HybridClock initialized: now 1786127109305992 us; error 42 us; skew 500 ppm
I20260807 18:25:09.306463 20046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:09 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127109, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:09.310235 20046 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:25:09.312100 20046 webserver.cc:460] Webserver started at http://127.29.156.194:35959/ using document root <none> and password file <none>
I20260807 18:25:09.312871 20046 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:09.313048 20046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:09.313396 20046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:25:09.316956 20046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "adaf21db71314f38b9e1481a95b97a58"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "30a863b4fe39a5356be6a9dbcb7cb358"
server_key_iv: "42c18e063eddc42cbdbd163358090154"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.318240 20046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "adaf21db71314f38b9e1481a95b97a58"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "30a863b4fe39a5356be6a9dbcb7cb358"
server_key_iv: "42c18e063eddc42cbdbd163358090154"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.327303 20046 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 18:25:09.331610 20065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.334148 20046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:25:09.334365 20046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "adaf21db71314f38b9e1481a95b97a58"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "30a863b4fe39a5356be6a9dbcb7cb358"
server_key_iv: "42c18e063eddc42cbdbd163358090154"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.334679 20046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:09.368551 20046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:09.373664 20046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:09.373975 20046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:09.375123 20046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:09.376937 20046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:09.377043 20046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.377290 20046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:09.377357 20046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.416137 20046 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:39501
I20260807 18:25:09.416250 20234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:39501 every 8 connection(s)
I20260807 18:25:09.417681 20046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:25:09.426550 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 20046
I20260807 18:25:09.426748 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:25:09.427273 30323 external_mini_cluster.cc:1428] Setting key 1a82499ed4138f1f41cc83f1e1569972
Aug 07 18:25:09 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127109, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:09.439908 20239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:09.420679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:34569 (local address 127.29.156.254:44025)
0807 18:25:09.421255 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:09.421259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:09.422361 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 18:25:09.424268 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:09.424273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:09.424344 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:09.424448 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:09.425949 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:09.426773 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:09.428632 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:09.429231 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:09.429372 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:09.433622 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:09.433628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:09.433631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:09.433665 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:09.436501 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:09.437307 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:09.437312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:09.437314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:09.437446 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:09.437930 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:09.437934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:09.437936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:09.438205 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:09.438366 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:09.439227 (+ 861us) server_negotiation.cc:299] Negotiation successful
0807 18:25:09.439462 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":329,"threads_started":1}
I20260807 18:25:09.440953 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:09.421425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:34569)
0807 18:25:09.422180 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:09.422220 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:09.423983 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:09.424673 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:09.424680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:09.425244 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:09.425793 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:09.425809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:09.426939 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:09.426944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:09.428467 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:09.428480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:09.429488 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:09.429495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:09.429807 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:09.430577 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:09.430618 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:09.433381 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:09.436659 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:09.436668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:09.436687 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:09.437157 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:09.437573 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:09.437578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:09.437582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:09.437791 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:09.438342 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:09.438351 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:09.438791 (+ 440us) client_negotiation.cc:769] Sending connection context
0807 18:25:09.439114 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 18:25:09.439725 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":300,"threads_started":1}
I20260807 18:25:09.442571 20237 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:09.442919 20237 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:09.443800 20237 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:09.445664 19754 ts_manager.cc:194] Registered new tserver with Master: adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
I20260807 18:25:09.446593 19754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:34569
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:25:09.503681 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:25:09.858604 20035 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
W20260807 18:25:09.861949 20243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:09.862262 20243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:09.862473 20243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:09.870891 20243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:09.871052 20243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:09.871276 20243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:25:09.882037 20243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:09.883435 20243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:09.884912 20243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:09.899184 20249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:09.899152 20250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:09.899232 20253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:09.900967 20243 server_base.cc:1034] running on GCE node
I20260807 18:25:09.901743 20243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:09.902781 20243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:09.904057 20243 hybrid_clock.cc:648] HybridClock initialized: now 1786127109903986 us; error 48 us; skew 500 ppm
I20260807 18:25:09.904409 20243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:09 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127109, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:09.907986 20243 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:25:09.909570 20243 webserver.cc:460] Webserver started at http://127.29.156.195:34439/ using document root <none> and password file <none>
I20260807 18:25:09.910234 20243 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:09.910338 20243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:09.910600 20243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:25:09.913699 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "d96fa2036a6e654c87b6ab11a63c41f2"
server_key_iv: "c344f5bf05a3eb4cb382614bbdd8e740"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.914430 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "d96fa2036a6e654c87b6ab11a63c41f2"
server_key_iv: "c344f5bf05a3eb4cb382614bbdd8e740"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.919812 20243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:25:09.923358 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.924991 20243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:25:09.925144 20243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "d96fa2036a6e654c87b6ab11a63c41f2"
server_key_iv: "c344f5bf05a3eb4cb382614bbdd8e740"
server_key_version: "encryptionkey@0"
I20260807 18:25:09.925335 20243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:09.948307 20243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:09.952611 20243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:09.952872 20243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:09.953850 20243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:09.955181 20243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:09.955262 20243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.955349 20243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:09.955410 20243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:09.993364 20243 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45787
I20260807 18:25:09.993412 20449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45787 every 8 connection(s)
I20260807 18:25:09.994810 20243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:25:09.997066 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 20243
I20260807 18:25:09.997303 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:25:09.997911 30323 external_mini_cluster.cc:1428] Setting key f345882940444f66ad9c813b8c166bd8
Aug 07 18:25:10 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127109, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:10.017277 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:09.997752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:38575 (local address 127.29.156.254:44025)
0807 18:25:09.998414 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:09.998421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:09.999584 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 18:25:10.001462 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.001471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.001558 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.001733 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:10.003497 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.004556 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.006264 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.006771 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.006869 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.010569 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:10.010575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:10.010579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:10.010631 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:10.013883 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.014709 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.014717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.014719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.014865 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.015434 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.015438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.015440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.015820 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:10.015985 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.016752 (+ 767us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.016973 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":339,"threads_started":1}
I20260807 18:25:10.018311 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:09.998384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:38575)
0807 18:25:09.999241 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:09.999298 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.001188 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.002077 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.002107 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.002694 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:10.003288 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.003307 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.004758 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.004762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.006088 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.006101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.006991 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.006997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.007199 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.007992 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:10.008017 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:10.010337 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:10.014032 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:10.014045 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:10.014065 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:10.014558 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:10.014989 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:10.014993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:10.014995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:10.015258 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:10.015956 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:10.015963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:10.016371 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.016634 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.017087 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":367,"threads_started":1}
I20260807 18:25:10.019905 20450 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:10.020251 20450 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:10.021015 20450 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:10.022532 19754 ts_manager.cc:194] Registered new tserver with Master: e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787)
I20260807 18:25:10.023387 19754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:38575
I20260807 18:25:10.026512 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:25:10.046979 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.031733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.29.156.254:44025)
0807 18:25:10.031911 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.031916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.032478 (+ 562us) server_negotiation.cc:407] Connection header received
0807 18:25:10.032677 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.032681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.032766 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.032859 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:10.034326 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.035103 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.036717 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.037161 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.037270 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.040672 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:10.040678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:10.040681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:10.040722 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:10.043832 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.044920 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.044924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.044926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.045121 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.045624 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.045627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.045629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.045969 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:10.046209 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.046561 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.046751 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:25:10.051774 19754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:25:10.059422 19754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:25:10.099062 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.074440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.29.156.195:45787)
0807 18:25:10.091071 (+ 16631us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.091083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.091128 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:25:10.091245 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.091257 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.091659 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.091908 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:10.092976 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.094239 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.096865 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.097827 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.098058 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.098278 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.098533 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.098706 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16421,"thread_start_us":16219,"threads_started":1}
I20260807 18:25:10.099555 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.073638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45787 (local address 127.0.0.1:44070)
0807 18:25:10.090064 (+ 16426us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.090101 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.090283 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.092242 (+ 1959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.092249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.092278 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:10.092746 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.092764 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.094389 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.094395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.096661 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.096677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.098064 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.098071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.098260 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.098317 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.098477 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.098684 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16209,"mutex_wait_us":79,"thread_start_us":12781,"threads_started":1}
I20260807 18:25:10.100348 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.073673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.29.156.194:39501)
0807 18:25:10.083889 (+ 10216us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.083903 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.087148 (+ 3245us) server_negotiation.cc:407] Connection header received
0807 18:25:10.087573 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.087585 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.087906 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.088206 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:10.090203 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.093434 (+ 3231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.096755 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.098442 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.098616 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.098801 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.099691 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.099875 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9931,"spinlock_wait_cycles":68352,"thread_start_us":9778,"threads_started":1}
I20260807 18:25:10.100811 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.073469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39501 (local address 127.0.0.1:40340)
0807 18:25:10.086924 (+ 13455us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.086990 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.087285 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.089369 (+ 2084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.089382 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.089438 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:10.089985 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.089998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.093595 (+ 3597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.093602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.096482 (+ 2880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.096506 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.098977 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.098991 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.099194 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.099246 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.099344 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.099482 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13133,"mutex_wait_us":4,"thread_start_us":17637,"threads_started":1}
I20260807 18:25:10.101833 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.074122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39598)
0807 18:25:10.091462 (+ 17340us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.091497 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.091794 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.092831 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.092839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.092873 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:10.093250 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.093259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.095442 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.095449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.097958 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.097980 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.100865 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.100873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.101140 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.101183 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.101282 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.101427 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17208,"mutex_wait_us":1,"thread_start_us":15519,"threads_started":1}
I20260807 18:25:10.103127 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.074869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.29.156.193:38015)
0807 18:25:10.089365 (+ 14496us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.089381 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.091650 (+ 2269us) server_negotiation.cc:407] Connection header received
0807 18:25:10.092035 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.092046 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.092295 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.092483 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:10.093438 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.095240 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.098181 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.100367 (+ 2186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.100826 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.102509 (+ 1683us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.102585 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.102751 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14209,"thread_start_us":13973,"threads_started":1}
I20260807 18:25:10.104517 20133 tablet_service.cc:1467] Processing CreateTablet for tablet 2270792a5e0a4790b6a3791dda2900ae (DEFAULT_TABLE table=test-table [id=0cd63ee9c4f0420e8221c0390fe238ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:10.104547 20355 tablet_service.cc:1467] Processing CreateTablet for tablet 2270792a5e0a4790b6a3791dda2900ae (DEFAULT_TABLE table=test-table [id=0cd63ee9c4f0420e8221c0390fe238ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:10.106088 20133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2270792a5e0a4790b6a3791dda2900ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:10.106222 20355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2270792a5e0a4790b6a3791dda2900ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:10.107151 19934 tablet_service.cc:1467] Processing CreateTablet for tablet 2270792a5e0a4790b6a3791dda2900ae (DEFAULT_TABLE table=test-table [id=0cd63ee9c4f0420e8221c0390fe238ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:10.108796 19934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2270792a5e0a4790b6a3791dda2900ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:10.119283 20479 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Bootstrap starting.
I20260807 18:25:10.122354 20479 tablet_bootstrap.cc:654] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:10.123015 20480 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Bootstrap starting.
I20260807 18:25:10.123860 20479 log.cc:824] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:10.125413 20483 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Bootstrap starting.
I20260807 18:25:10.127061 20480 tablet_bootstrap.cc:654] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:10.127562 20479 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: No bootstrap required, opened a new log
I20260807 18:25:10.127859 20479 ts_tablet_manager.cc:1397] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260807 18:25:10.128408 20480 log.cc:824] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:10.130189 20483 tablet_bootstrap.cc:654] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:10.131314 20483 log.cc:824] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:10.131866 20480 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: No bootstrap required, opened a new log
I20260807 18:25:10.132136 20480 ts_tablet_manager.cc:1397] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260807 18:25:10.134331 20483 tablet_bootstrap.cc:492] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: No bootstrap required, opened a new log
I20260807 18:25:10.134651 20483 ts_tablet_manager.cc:1397] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 18:25:10.139834 20479 raft_consensus.cc:348] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.140357 20479 raft_consensus.cc:374] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:10.140461 20479 raft_consensus.cc:729] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adaf21db71314f38b9e1481a95b97a58, State: Initialized, Role: FOLLOWER
I20260807 18:25:10.141240 20479 consensus_queue.cc:260] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.142547 20237 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
I20260807 18:25:10.143292 20479 ts_tablet_manager.cc:1428] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260807 18:25:10.143767 20480 raft_consensus.cc:348] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.144215 20480 raft_consensus.cc:374] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:10.144328 20480 raft_consensus.cc:729] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21d076c394e46ee96c4aeddc9cd4c97, State: Initialized, Role: FOLLOWER
I20260807 18:25:10.144977 20480 consensus_queue.cc:260] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.146229 20450 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
I20260807 18:25:10.146369 20483 raft_consensus.cc:348] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.146823 20483 raft_consensus.cc:374] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:10.146929 20480 ts_tablet_manager.cc:1428] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Time spent starting tablet: real 0.015s user 0.004s sys 0.010s
I20260807 18:25:10.146924 20483 raft_consensus.cc:729] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f5894b2402434ead123b469c69a85e, State: Initialized, Role: FOLLOWER
I20260807 18:25:10.147663 20483 consensus_queue.cc:260] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.149329 20483 ts_tablet_manager.cc:1428] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260807 18:25:10.150839 20490 raft_consensus.cc:482] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:25:10.151108 20490 raft_consensus.cc:504] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.152985 20490 leader_election.cc:288] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501), e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787)
I20260807 18:25:10.163666 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.156199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:39517 (local address 127.29.156.194:39501)
0807 18:25:10.156413 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.156422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.157385 (+ 963us) server_negotiation.cc:407] Connection header received
0807 18:25:10.157659 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.157665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.157750 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.157817 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:10.158717 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.159757 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.161710 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.162599 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.162756 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.162832 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.163208 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.163346 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:25:10.163848 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.155954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39501 (local address 127.29.156.193:39517)
0807 18:25:10.157283 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.157300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.157482 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.158049 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.158054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.158078 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:10.158506 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.158517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.159901 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.159907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.161539 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.161551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.162826 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.162833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.163018 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.163058 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.163176 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.163327 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":60,"thread_start_us":475,"threads_started":1}
I20260807 18:25:10.164851 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.153449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:57343 (local address 127.29.156.195:45787)
0807 18:25:10.153663 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.153668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.156895 (+ 3227us) server_negotiation.cc:407] Connection header received
0807 18:25:10.157231 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.157238 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.157313 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.157375 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:10.158218 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.159506 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.162518 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.163682 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.163854 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.163920 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.164464 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.164598 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:25:10.164949 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.153308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45787 (local address 127.29.156.193:57343)
0807 18:25:10.156786 (+ 3478us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.156808 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:10.157033 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:10.157595 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:10.157600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:10.157628 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:10.158008 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.158023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.159733 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.159739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.162314 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:10.162331 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.164093 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.164099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:10.164287 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.164328 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:25:10.164436 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:25:10.164563 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3338,"mutex_wait_us":47,"thread_start_us":5992,"threads_started":1}
I20260807 18:25:10.165163 20163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2270792a5e0a4790b6a3791dda2900ae" candidate_uuid: "95f5894b2402434ead123b469c69a85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adaf21db71314f38b9e1481a95b97a58" is_pre_election: true
I20260807 18:25:10.165761 20163 raft_consensus.cc:2454] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f5894b2402434ead123b469c69a85e in term 0.
I20260807 18:25:10.165874 20383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2270792a5e0a4790b6a3791dda2900ae" candidate_uuid: "95f5894b2402434ead123b469c69a85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" is_pre_election: true
I20260807 18:25:10.166388 20383 raft_consensus.cc:2454] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f5894b2402434ead123b469c69a85e in term 0.
I20260807 18:25:10.166497 19869 leader_election.cc:302] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f5894b2402434ead123b469c69a85e, adaf21db71314f38b9e1481a95b97a58; no voters:
I20260807 18:25:10.166958 20490 raft_consensus.cc:2781] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:25:10.167135 20490 raft_consensus.cc:482] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:25:10.167276 20490 raft_consensus.cc:3037] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:25:10.170018 20238 tablet.cc:2369] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:25:10.170150 20490 raft_consensus.cc:504] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.170974 20490 leader_election.cc:288] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 1 election: Requested vote from peers adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501), e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787)
I20260807 18:25:10.171389 20163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2270792a5e0a4790b6a3791dda2900ae" candidate_uuid: "95f5894b2402434ead123b469c69a85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adaf21db71314f38b9e1481a95b97a58"
I20260807 18:25:10.171635 20163 raft_consensus.cc:3037] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:10.171902 20383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2270792a5e0a4790b6a3791dda2900ae" candidate_uuid: "95f5894b2402434ead123b469c69a85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
I20260807 18:25:10.172152 20383 raft_consensus.cc:3037] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:10.173959 20163 raft_consensus.cc:2454] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f5894b2402434ead123b469c69a85e in term 1.
I20260807 18:25:10.175695 20383 raft_consensus.cc:2454] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f5894b2402434ead123b469c69a85e in term 1.
I20260807 18:25:10.175748 19869 leader_election.cc:302] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f5894b2402434ead123b469c69a85e, adaf21db71314f38b9e1481a95b97a58; no voters:
I20260807 18:25:10.176098 20490 raft_consensus.cc:2781] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:25:10.176635 20490 raft_consensus.cc:686] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 1 LEADER]: Becoming Leader. State: Replica: 95f5894b2402434ead123b469c69a85e, State: Running, Role: LEADER
I20260807 18:25:10.177162 20490 consensus_queue.cc:237] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:10.182400 19754 catalog_manager.cc:5495] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e reported cstate change: term changed from 0 to 1, leader changed from <none> to 95f5894b2402434ead123b469c69a85e (127.29.156.193). New cstate: current_term: 1 leader_uuid: "95f5894b2402434ead123b469c69a85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } health_report { overall_health: HEALTHY } } }
I20260807 18:25:10.202366 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.196499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.29.156.193:38015)
0807 18:25:10.196744 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.196753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.196839 (+ 86us) server_negotiation.cc:407] Connection header received
0807 18:25:10.197091 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.197096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.197175 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.197338 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:25:10.197741 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.198506 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.200078 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.200489 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.200637 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.201215 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:25:10.201760 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:25:10.201941 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.201993 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.202077 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:25:10.211030 20163 raft_consensus.cc:1261] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Refusing update from remote peer 95f5894b2402434ead123b469c69a85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:25:10.211030 20383 raft_consensus.cc:1261] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Refusing update from remote peer 95f5894b2402434ead123b469c69a85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:25:10.212075 20496 consensus_queue.cc:1035] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:25:10.212492 20490 consensus_queue.cc:1035] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:25:10.238430 19754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:25:10.238750 19754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:25:10.242959 19754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Sending DeleteTablet for 3 replicas of tablet 2270792a5e0a4790b6a3791dda2900ae
I20260807 18:25:10.244215 20133 tablet_service.cc:1514] Processing DeleteTablet for tablet 2270792a5e0a4790b6a3791dda2900ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:40340
I20260807 18:25:10.244220 20355 tablet_service.cc:1514] Processing DeleteTablet for tablet 2270792a5e0a4790b6a3791dda2900ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:44070
I20260807 18:25:10.244436 19934 tablet_service.cc:1514] Processing DeleteTablet for tablet 2270792a5e0a4790b6a3791dda2900ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:10 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:39598
I20260807 18:25:10.245759 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 19838
I20260807 18:25:10.254966 20517 tablet_replica.cc:330] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: stopping tablet replica
I20260807 18:25:10.255473 20517 raft_consensus.cc:2229] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:25:10.256067 20517 raft_consensus.cc:2258] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:10.260193 20517 ts_tablet_manager.cc:1905] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:10.266748 20516 tablet_replica.cc:330] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: stopping tablet replica
I20260807 18:25:10.267243 20516 raft_consensus.cc:2229] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:25:10.267805 20516 raft_consensus.cc:2258] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:10.270902 20518 tablet_replica.cc:330] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: stopping tablet replica
I20260807 18:25:10.271849 20518 raft_consensus.cc:2229] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:25:10.272279 20516 ts_tablet_manager.cc:1905] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:10.272558 20517 ts_tablet_manager.cc:1918] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:25:10.272720 20517 log.cc:1196] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/2270792a5e0a4790b6a3791dda2900ae
I20260807 18:25:10.273295 20517 ts_tablet_manager.cc:1939] T 2270792a5e0a4790b6a3791dda2900ae P e21d076c394e46ee96c4aeddc9cd4c97: Deleting consensus metadata
I20260807 18:25:10.274739 20518 raft_consensus.cc:2258] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:10.275290 19731 catalog_manager.cc:4841] TS e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): tablet 2270792a5e0a4790b6a3791dda2900ae (table test-table [id=0cd63ee9c4f0420e8221c0390fe238ed]) successfully deleted
Aug 07 18:25:10 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:25:10.280226 20518 ts_tablet_manager.cc:1905] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:10.280298 20516 ts_tablet_manager.cc:1918] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:25:10.280463 20516 log.cc:1196] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/2270792a5e0a4790b6a3791dda2900ae
I20260807 18:25:10.281064 20516 ts_tablet_manager.cc:1939] T 2270792a5e0a4790b6a3791dda2900ae P adaf21db71314f38b9e1481a95b97a58: Deleting consensus metadata
I20260807 18:25:10.283169 19730 catalog_manager.cc:4841] TS adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): tablet 2270792a5e0a4790b6a3791dda2900ae (table test-table [id=0cd63ee9c4f0420e8221c0390fe238ed]) successfully deleted
I20260807 18:25:10.288619 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.251135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.29.156.193:38015)
0807 18:25:10.251382 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.251391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.261893 (+ 10502us) server_negotiation.cc:407] Connection header received
0807 18:25:10.262492 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.262502 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.262639 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.262740 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:10.265682 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.267368 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.272108 (+ 4740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.272839 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.272959 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.280109 (+ 7150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:10.280130 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:10.280149 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:10.280196 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:10.284834 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.285881 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.285892 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.285895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.286115 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.286634 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.286643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.286646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.287183 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:10.287555 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.287939 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.288213 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:25:10.288894 20518 ts_tablet_manager.cc:1918] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:25:10.289098 20518 log.cc:1196] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/2270792a5e0a4790b6a3791dda2900ae
I20260807 18:25:10.289417 20022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:10.392508 20518 ts_tablet_manager.cc:1939] T 2270792a5e0a4790b6a3791dda2900ae P 95f5894b2402434ead123b469c69a85e: Deleting consensus metadata
I20260807 18:25:10.393153 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 19838
I20260807 18:25:10.421072 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 20046
W20260807 18:25:10.420864 19732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.156.193:38015 (error 108)
I20260807 18:25:10.423357 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.422851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39628)
0807 18:25:10.423033 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.423159 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:25:10.425001 19732 catalog_manager.cc:4584] TS 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2270792a5e0a4790b6a3791dda2900ae: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Aug 07 18:25:10 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:25:10.441061 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.422526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.29.156.194:39501)
0807 18:25:10.422728 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.422734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.422756 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:25:10.422820 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.422824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.422898 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.423029 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:10.424855 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.426027 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.427608 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.428290 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.428407 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.433549 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:10.433562 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:10.433579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:10.433625 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:10.437535 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.438455 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.438494 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.438497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.438645 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.439117 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.439123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.439128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.439669 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:10.440031 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.440380 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.440649 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:25:10.441821 20228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:10.478765 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.478248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39642)
0807 18:25:10.478481 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.478596 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 18:25:10.480077 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.479675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39654)
0807 18:25:10.479845 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.479937 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:25:10.535283 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 20046
I20260807 18:25:10.541024 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.540393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39660)
0807 18:25:10.540683 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.540798 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:25:10.542327 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.541828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39668)
0807 18:25:10.542080 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.542183 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:25:10.563951 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 20243
Aug 07 18:25:10 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:25:10.580724 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.564874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.29.156.195:45787)
0807 18:25:10.565082 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:10.565087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:10.565106 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:25:10.565258 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:10.565262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:10.565330 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:10.565417 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:10.566924 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.567812 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.569184 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:10.569619 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:10.569763 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:10.574554 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:10.574571 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:10.574588 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:10.574627 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:10.577745 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.578512 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.578520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.578522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.578690 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:10.579064 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:10.579067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:10.579069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:10.579447 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:10.579773 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:10.580161 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 18:25:10.580339 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:25:10.581403 20438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:10.623276 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.622734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39680)
0807 18:25:10.622986 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.623108 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:25:10.624492 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.624020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39684)
0807 18:25:10.624279 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.624353 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 18:25:10.673636 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 20243
I20260807 18:25:10.698047 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 19702
I20260807 18:25:10.699438 19793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:10.814718 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:10.813874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:39690)
0807 18:25:10.814119 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:10.814250 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:25:10.815335 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 19702
I20260807 18:25:30.847474 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--webserver_interface=127.29.156.254
--webserver_port=46563
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:25:31.218788 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:31.219137 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:31.219244 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:31.227469 20527 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:25:31.227592 20527 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:25:31.227635 20527 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:25:31.227687 20527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:25:31.227743 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:31.227813 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:31.227860 20527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:25:31.227911 20527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:25:31.238291 20527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=46563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:31.239661 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:31.241017 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:31.269163 20543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:31.269263 20537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:31.269299 20539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:31.270098 20527 server_base.cc:1034] running on GCE node
I20260807 18:25:31.271207 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:31.273015 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:31.274277 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786127131274198 us; error 46 us; skew 500 ppm
I20260807 18:25:31.274696 20527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:31 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:31.283248 20527 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:25:31.285450 20527 webserver.cc:460] Webserver started at http://127.29.156.254:46563/ using document root <none> and password file <none>
I20260807 18:25:31.286396 20527 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:31.286556 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:31.294142 20527 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:25:31.297713 20556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:31.299371 20527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:25:31.299639 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "cdd7031236594757b48dcb1fa09a9442"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "63588b8c50ddde6abf1edf1239eba542"
server_key_iv: "243fd0af60eaafadebb73f415285ca12"
server_key_version: "encryptionkey@0"
I20260807 18:25:31.300861 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:31.341048 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:31.346174 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:31.346556 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:31.369258 20527 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44025
I20260807 18:25:31.369333 20633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44025 every 8 connection(s)
I20260807 18:25:31.371435 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:25:31.376601 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 20527
I20260807 18:25:31.377700 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:38015
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=43795
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:25:31.380414 20634 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:25:31.386147 20634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap starting.
I20260807 18:25:31.409241 20634 log.cc:824] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:31.423902 20634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:25:31.424511 20634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap complete.
I20260807 18:25:31.434576 20634 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:31.435565 20634 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Initialized, Role: FOLLOWER
I20260807 18:25:31.436218 20634 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:31.436470 20634 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:25:31.436630 20634 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:25:31.436829 20634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:25:31.440868 20634 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:31.441514 20634 leader_election.cc:302] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdd7031236594757b48dcb1fa09a9442; no voters:
I20260807 18:25:31.442203 20634 leader_election.cc:288] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:25:31.442353 20640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:25:31.443629 20640 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 LEADER]: Becoming Leader. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Running, Role: LEADER
I20260807 18:25:31.444224 20640 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:31.444710 20634 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:25:31.447507 20641 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:31.447484 20642 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd7031236594757b48dcb1fa09a9442. Latest consensus state: current_term: 2 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:31.448107 20641 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:31.448138 20642 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:31.449213 20648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:25:31.471354 20648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:25:31.473845 20648 catalog_manager.cc:1259] Loaded cluster ID: 775ac5c0b9fd444f9b8a2cbc64907e7d
I20260807 18:25:31.474052 20648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:25:31.478929 20648 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:25:31.480907 20648 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Loaded TSK: 0
I20260807 18:25:31.521865 20663 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:25:31.525254 20648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Generated new TSK 1
I20260807 18:25:31.525863 20648 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:25:31.776780 20638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:31.777088 20638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:31.777251 20638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:31.784886 20638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:31.785060 20638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:31.785270 20638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:25:31.795750 20638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:38015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=43795
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:31.797695 20638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:31.799237 20638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:31.815851 20677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:31.816197 20673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:31.815981 20674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:31.817747 20638 server_base.cc:1034] running on GCE node
I20260807 18:25:31.818544 20638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:31.819730 20638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:31.821044 20638 hybrid_clock.cc:648] HybridClock initialized: now 1786127131820990 us; error 42 us; skew 500 ppm
I20260807 18:25:31.821422 20638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:31 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:31.825276 20638 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:25:31.827090 20638 webserver.cc:460] Webserver started at http://127.29.156.193:43795/ using document root <none> and password file <none>
I20260807 18:25:31.828018 20638 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:31.828256 20638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:31.834623 20638 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:25:31.837958 20688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:31.839757 20638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:25:31.839972 20638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "95f5894b2402434ead123b469c69a85e"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "bcdad0ed977377cf1062bcee3b2b9266"
server_key_iv: "8c4e6032242a3085146f8511d5dc898e"
server_key_version: "encryptionkey@0"
I20260807 18:25:31.841039 20638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:31.865999 20638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:31.870822 20638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:31.871135 20638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:31.872283 20638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:31.873881 20638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:31.874023 20638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:31.874162 20638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:31.874264 20638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:31.911788 20638 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38015
I20260807 18:25:31.911881 20854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38015 every 8 connection(s)
I20260807 18:25:31.913305 20638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:25:31.915066 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 20638
I20260807 18:25:31.916003 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:39501
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=35959
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:25:31 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:31.941977 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:31.917041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:60153 (local address 127.29.156.254:44025)
0807 18:25:31.917772 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:31.917792 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:31.918457 (+ 665us) server_negotiation.cc:407] Connection header received
0807 18:25:31.920835 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:31.920857 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:31.921369 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:31.921831 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:31.923335 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:31.925703 (+ 2368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:31.927442 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:31.928738 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:31.928951 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:31.933616 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:31.933625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:31.933643 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:31.933685 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:31.937023 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:31.937869 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:31.937877 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:31.937883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:31.938073 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:31.938563 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:31.938568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:31.938573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:31.939094 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:31.939300 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:31.940200 (+ 900us) server_negotiation.cc:299] Negotiation successful
0807 18:25:31.940598 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":297,"threads_started":1}
I20260807 18:25:31.941977 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:31.917304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:60153)
0807 18:25:31.918235 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:31.918290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:31.920209 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:31.921974 (+ 1765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:31.921982 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:31.922522 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:31.923082 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:31.923099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:31.925906 (+ 2807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:31.925910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:31.927254 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:31.927279 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:31.929393 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:31.929404 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:31.929782 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:31.930676 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:31.930701 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:31.933274 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:31.937203 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:31.937213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:31.937230 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:31.937706 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:31.938213 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:31.938219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:31.938223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:31.938421 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:31.939252 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:31.939260 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:31.939659 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 18:25:31.939948 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:25:31.940578 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":394,"threads_started":1}
I20260807 18:25:31.945555 20856 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:31.945982 20856 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:31.946909 20856 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:31.949584 20580 ts_manager.cc:194] Registered new tserver with Master: 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015)
I20260807 18:25:31.951989 20580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:60153
W20260807 18:25:32.286892 20860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:32.287225 20860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:32.287325 20860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:32.295135 20860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:32.295270 20860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:32.295392 20860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:25:32.306097 20860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:39501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=35959
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:32.307435 20860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:32.308724 20860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:32.323302 20879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:32.323257 20875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:32.323349 20874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:32.329628 20860 server_base.cc:1034] running on GCE node
I20260807 18:25:32.330370 20860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:32.331641 20860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:32.332916 20860 hybrid_clock.cc:648] HybridClock initialized: now 1786127132332810 us; error 95 us; skew 500 ppm
I20260807 18:25:32.333298 20860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:32 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127132, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:32.337095 20860 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:25:32.338829 20860 webserver.cc:460] Webserver started at http://127.29.156.194:35959/ using document root <none> and password file <none>
I20260807 18:25:32.339666 20860 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:32.339802 20860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:32.346213 20860 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:25:32.349504 20886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.351257 20860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:25:32.351497 20860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "adaf21db71314f38b9e1481a95b97a58"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "30a863b4fe39a5356be6a9dbcb7cb358"
server_key_iv: "42c18e063eddc42cbdbd163358090154"
server_key_version: "encryptionkey@0"
I20260807 18:25:32.352710 20860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:32.371829 20860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:32.376730 20860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:32.377058 20860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:32.378291 20860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:32.379844 20860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:32.379945 20860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.380123 20860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:32.380219 20860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.420287 20860 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:39501
I20260807 18:25:32.420399 21045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:39501 every 8 connection(s)
I20260807 18:25:32.422336 20860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:25:32.423444 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 20860
I20260807 18:25:32.424300 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:45787
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=34439
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:25:32 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127132, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:32.450224 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.426244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:40835 (local address 127.29.156.254:44025)
0807 18:25:32.426468 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.426475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.427960 (+ 1485us) server_negotiation.cc:407] Connection header received
0807 18:25:32.430825 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.430833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.430968 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.431082 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:32.433342 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.434342 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.436929 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.437667 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.437800 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.442325 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:32.442334 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:32.442342 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:32.442392 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:32.446396 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.447261 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.447269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.447276 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.447435 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.447955 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.447963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.447968 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.448383 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:32.448541 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.449456 (+ 915us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.449744 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:25:32.451066 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.426805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:40835)
0807 18:25:32.427740 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:32.427801 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:32.430506 (+ 2705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:32.431369 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:32.431383 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:32.432281 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:32.433134 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.433163 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.434533 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.434541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.436773 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.436792 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.437923 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.437933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.438289 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.439263 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:32.439288 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:32.442025 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:32.446575 (+ 4550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:32.446584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:32.446598 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:32.447106 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:32.447606 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:32.447610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:32.447612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:32.447820 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:32.448551 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:32.448559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:32.449028 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 18:25:32.449306 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 18:25:32.449788 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":330,"threads_started":1}
I20260807 18:25:32.452726 21046 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:32.453136 21046 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:32.454077 21046 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:32.455945 20580 ts_manager.cc:194] Registered new tserver with Master: adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
I20260807 18:25:32.457031 20580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:40835
W20260807 18:25:32.780049 21049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:32.780329 21049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:32.780478 21049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:32.787990 21049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:32.788121 21049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:32.788305 21049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:25:32.798998 21049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:45787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=34439
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:32.800413 21049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:32.801818 21049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:32.815907 21054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:32.815968 21058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:32.815954 21055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:32.817055 21049 server_base.cc:1034] running on GCE node
I20260807 18:25:32.817783 21049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:32.818719 21049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:32.819926 21049 hybrid_clock.cc:648] HybridClock initialized: now 1786127132819854 us; error 57 us; skew 500 ppm
I20260807 18:25:32.820276 21049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:32 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127132, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:32.823982 21049 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:25:32.825742 21049 webserver.cc:460] Webserver started at http://127.29.156.195:34439/ using document root <none> and password file <none>
I20260807 18:25:32.826460 21049 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:32.826596 21049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:32.832818 21049 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:25:32.835736 21070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.837309 21049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:25:32.837548 21049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "d96fa2036a6e654c87b6ab11a63c41f2"
server_key_iv: "c344f5bf05a3eb4cb382614bbdd8e740"
server_key_version: "encryptionkey@0"
I20260807 18:25:32.838660 21049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:32.858790 21049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:32.863334 21049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:32.863699 21049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:32.864718 21049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:32.866091 21049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:32.866240 21049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.866372 21049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:32.866466 21049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:32.902966 21049 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45787
I20260807 18:25:32.903093 21235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45787 every 8 connection(s)
I20260807 18:25:32.904542 21049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:25:32.914719 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 21049
Aug 07 18:25:32 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127132, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:25:32.930636 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.907572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:35999 (local address 127.29.156.254:44025)
0807 18:25:32.907786 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.907792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.909089 (+ 1297us) server_negotiation.cc:407] Connection header received
0807 18:25:32.910948 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.910954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.911055 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.911191 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:32.912882 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.914212 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.916905 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.917864 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.918000 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.924148 (+ 6148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:32.924154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:32.924157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:32.924197 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:32.927161 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.928059 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.928065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.928068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.928211 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.928684 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.928688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.928690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.929065 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:32.929223 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.930010 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.930237 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:25:32.931780 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.908085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:35999)
0807 18:25:32.908899 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:32.908952 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:32.910648 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:32.911448 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:32.911457 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:32.912122 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:32.912680 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.912697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.914439 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.914445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.916676 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.916699 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.918963 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.918973 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.919445 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.920565 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:32.920682 (+ 117us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:32.923902 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:32.927333 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:32.927344 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:32.927363 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:32.927926 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:32.928330 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:32.928335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:32.928339 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:32.928551 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:32.929197 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:32.929205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:32.929599 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 18:25:32.929898 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:25:32.930481 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":359,"threads_started":1}
I20260807 18:25:32.933576 21236 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44025
I20260807 18:25:32.933868 21236 heartbeater.cc:464] Registering TS with master...
I20260807 18:25:32.934789 21236 heartbeater.cc:511] Master 127.29.156.254:44025 requested a full tablet report, sending...
I20260807 18:25:32.935925 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.915979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.29.156.254:44025)
0807 18:25:32.920183 (+ 4204us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.920190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.920765 (+ 575us) server_negotiation.cc:407] Connection header received
0807 18:25:32.921006 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.921011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.921084 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.921182 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:32.923353 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.924232 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.925637 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.926222 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.926399 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.929563 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:32.929569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:32.929572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:32.929602 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:32.932768 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.933576 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.933583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.933587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.933766 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.934249 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.934254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.934258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.934563 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:32.934837 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.935241 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.935492 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4034,"thread_start_us":3844,"threads_started":1}
I20260807 18:25:32.936496 20580 ts_manager.cc:194] Registered new tserver with Master: e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787)
I20260807 18:25:32.937633 20580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:35999
I20260807 18:25:32.938735 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:25:32.946471 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.939863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.29.156.254:44025)
0807 18:25:32.940035 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.940040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.940523 (+ 483us) server_negotiation.cc:407] Connection header received
0807 18:25:32.940683 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.940688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.940864 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.941010 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:25:32.941448 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.942134 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.944139 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.944754 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.944859 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.945456 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:25:32.945804 (+ 348us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:25:32.946205 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38072: token expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:25:32.946640 21244 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38072: token expired
W20260807 18:25:32.947242 20457 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.156.254:44025)
I20260807 18:25:32.955588 20856 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
I20260807 18:25:32.961159 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.947750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.29.156.254:44025)
0807 18:25:32.947905 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.947909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.947995 (+ 86us) server_negotiation.cc:407] Connection header received
0807 18:25:32.948107 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.948111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.948164 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.948238 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:32.949612 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.950391 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.951848 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.952461 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.952576 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.955405 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:32.955410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:32.955413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:32.955455 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:32.958560 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.959311 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.959314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.959317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.959466 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:32.959918 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:32.959923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:32.959925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:32.960226 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:32.960412 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.960723 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.960907 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:25:32.964884 20577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:25:32.972502 20577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:25:32.999125 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.986342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.29.156.194:39501)
0807 18:25:32.988863 (+ 2521us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.988876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.988916 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:25:32.989056 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.989069 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.989398 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.991662 (+ 2264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:32.991900 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.993239 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.996198 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.997719 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.997914 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.998130 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.998569 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.998762 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2331,"thread_start_us":2228,"threads_started":1}
I20260807 18:25:32.999199 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.985011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39501 (local address 127.0.0.1:53110)
0807 18:25:32.985790 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:32.985833 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:32.986045 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:32.990477 (+ 4432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:32.990486 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:32.990540 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:32.991052 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.991065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.993397 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.993402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.995971 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.995989 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.998016 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.998024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.998272 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.998344 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:25:32.998514 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:25:32.998741 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"spinlock_wait_cycles":5504,"thread_start_us":2568,"threads_started":1}
I20260807 18:25:32.999495 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.985399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.29.156.195:45787)
0807 18:25:32.990827 (+ 5428us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.990836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.990870 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:25:32.990988 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.990998 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.991262 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.991472 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:32.992195 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.994109 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.996812 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.997877 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.998098 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.998295 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:32.998965 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 18:25:32.999139 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5240,"thread_start_us":5059,"threads_started":1}
I20260807 18:25:33.000196 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.985029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45787 (local address 127.0.0.1:60304)
0807 18:25:32.990139 (+ 5110us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:32.990165 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:32.990545 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:32.991493 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:32.991498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:32.991559 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:32.991973 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.991990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.994253 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.994265 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.996647 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.996660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.998315 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.998321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.998497 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:32.998530 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:25:32.998613 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:25:32.998768 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4941,"mutex_wait_us":2,"spinlock_wait_cycles":1792,"thread_start_us":5141,"threads_started":1}
I20260807 18:25:33.001600 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.985872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52128)
0807 18:25:32.991369 (+ 5497us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:32.991389 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:32.992255 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:32.993274 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:32.993280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:32.993308 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:32.993840 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.993852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.995861 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.995868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:32.997967 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:32.997988 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.000709 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.000715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:33.000914 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.000965 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 18:25:33.001068 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:25:33.001197 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5370,"mutex_wait_us":4,"thread_start_us":3928,"threads_started":1}
I20260807 18:25:33.001730 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:32.986170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.29.156.193:38015)
0807 18:25:32.986999 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:32.987013 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:32.991503 (+ 4490us) server_negotiation.cc:407] Connection header received
0807 18:25:32.992521 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:32.992531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:32.992790 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:32.992981 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:32.994048 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.995689 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:32.998155 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:32.999819 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.000006 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.000180 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.001145 (+ 965us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.001346 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":440,"threads_started":1}
I20260807 18:25:33.004194 21147 tablet_service.cc:1467] Processing CreateTablet for tablet 432b26b59f48443683fbd805811e36be (DEFAULT_TABLE table=test-table [id=6532de5dd9274a789532e00e9ce68fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:33.004166 20950 tablet_service.cc:1467] Processing CreateTablet for tablet 432b26b59f48443683fbd805811e36be (DEFAULT_TABLE table=test-table [id=6532de5dd9274a789532e00e9ce68fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:33.005704 20950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432b26b59f48443683fbd805811e36be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:33.006198 21147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432b26b59f48443683fbd805811e36be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:33.005712 20757 tablet_service.cc:1467] Processing CreateTablet for tablet 432b26b59f48443683fbd805811e36be (DEFAULT_TABLE table=test-table [id=6532de5dd9274a789532e00e9ce68fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:25:33.007164 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432b26b59f48443683fbd805811e36be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:25:33.021091 21265 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Bootstrap starting.
I20260807 18:25:33.024739 21265 tablet_bootstrap.cc:654] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:33.025983 21265 log.cc:824] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:33.026840 21271 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Bootstrap starting.
I20260807 18:25:33.028059 21266 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Bootstrap starting.
I20260807 18:25:33.030961 21271 tablet_bootstrap.cc:654] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:33.031580 21265 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: No bootstrap required, opened a new log
I20260807 18:25:33.031878 21265 ts_tablet_manager.cc:1397] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:25:33.032087 21266 tablet_bootstrap.cc:654] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Neither blocks nor log segments found. Creating new log.
I20260807 18:25:33.032742 21271 log.cc:824] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:33.033767 21266 log.cc:824] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:33.036001 21271 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: No bootstrap required, opened a new log
I20260807 18:25:33.036278 21271 ts_tablet_manager.cc:1397] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:25:33.037149 21266 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: No bootstrap required, opened a new log
I20260807 18:25:33.037439 21266 ts_tablet_manager.cc:1397] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 18:25:33.044239 21265 raft_consensus.cc:348] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.044406 21271 raft_consensus.cc:348] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.044902 21265 raft_consensus.cc:374] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:33.044966 21271 raft_consensus.cc:374] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:33.045030 21265 raft_consensus.cc:729] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21d076c394e46ee96c4aeddc9cd4c97, State: Initialized, Role: FOLLOWER
I20260807 18:25:33.045100 21271 raft_consensus.cc:729] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f5894b2402434ead123b469c69a85e, State: Initialized, Role: FOLLOWER
I20260807 18:25:33.045763 21271 consensus_queue.cc:260] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.045887 21265 consensus_queue.cc:260] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.047749 21271 ts_tablet_manager.cc:1428] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 18:25:33.048765 21236 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
I20260807 18:25:33.049461 21265 ts_tablet_manager.cc:1428] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260807 18:25:33.051170 21266 raft_consensus.cc:348] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.051743 21266 raft_consensus.cc:374] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:25:33.051872 21266 raft_consensus.cc:729] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adaf21db71314f38b9e1481a95b97a58, State: Initialized, Role: FOLLOWER
I20260807 18:25:33.052624 21266 consensus_queue.cc:260] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.053944 21046 heartbeater.cc:503] Master 127.29.156.254:44025 was elected leader, sending a full tablet report...
I20260807 18:25:33.054822 21266 ts_tablet_manager.cc:1428] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
W20260807 18:25:33.156687 21238 tablet.cc:2369] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:25:33.165947 20857 tablet.cc:2369] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:25:33.173548 21276 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:25:33.173916 21276 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.175928 21276 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501), 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015)
W20260807 18:25:33.176505 21047 tablet.cc:2369] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:25:33.182998 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.176225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39501 (local address 127.29.156.195:59727)
0807 18:25:33.176562 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.176589 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:33.176730 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:33.177668 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:33.177673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:33.177702 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:33.177886 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:33.177894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.178943 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.178953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:33.180387 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:33.180402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.182218 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.182225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:33.182424 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.182461 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:25:33.182565 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:25:33.182705 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 18:25:33.182940 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.177120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:59727 (local address 127.29.156.194:39501)
0807 18:25:33.177345 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.177352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.177373 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:25:33.177452 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.177456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.177514 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.177631 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:33.178038 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.178785 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.180582 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.182029 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.182154 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.182216 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.182616 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.182726 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:25:33.184237 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.176290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.29.156.195:37323)
0807 18:25:33.177284 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.177301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:33.177495 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:33.177880 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:33.177885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:33.177911 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:25:33.178302 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:33.178312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.179471 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.179479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:33.181813 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:33.181830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.183433 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.183441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:33.183648 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.183671 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:25:33.183775 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:25:33.183898 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":575,"threads_started":1}
I20260807 18:25:33.184198 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.176629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:37323 (local address 127.29.156.193:38015)
0807 18:25:33.176857 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.176866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.177420 (+ 554us) server_negotiation.cc:407] Connection header received
0807 18:25:33.177665 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.177669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.177733 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.177861 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:25:33.178505 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.179308 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.181990 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.183185 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.183335 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.183419 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.183814 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.183945 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:25:33.184170 20979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432b26b59f48443683fbd805811e36be" candidate_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adaf21db71314f38b9e1481a95b97a58" is_pre_election: true
I20260807 18:25:33.184899 20979 raft_consensus.cc:2454] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e21d076c394e46ee96c4aeddc9cd4c97 in term 0.
I20260807 18:25:33.185696 21072 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97; no voters:
I20260807 18:25:33.185575 20789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432b26b59f48443683fbd805811e36be" candidate_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f5894b2402434ead123b469c69a85e" is_pre_election: true
I20260807 18:25:33.186172 21276 raft_consensus.cc:2781] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:25:33.186185 20789 raft_consensus.cc:2454] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e21d076c394e46ee96c4aeddc9cd4c97 in term 0.
I20260807 18:25:33.186347 21276 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:25:33.186514 21276 raft_consensus.cc:3037] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:33.189157 21276 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.189971 21276 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [CANDIDATE]: Term 1 election: Requested vote from peers adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501), 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015)
I20260807 18:25:33.190547 20789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432b26b59f48443683fbd805811e36be" candidate_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f5894b2402434ead123b469c69a85e"
I20260807 18:25:33.190567 20979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432b26b59f48443683fbd805811e36be" candidate_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adaf21db71314f38b9e1481a95b97a58"
I20260807 18:25:33.190793 20979 raft_consensus.cc:3037] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:33.190793 20789 raft_consensus.cc:3037] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:25:33.194582 20979 raft_consensus.cc:2454] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e21d076c394e46ee96c4aeddc9cd4c97 in term 1.
I20260807 18:25:33.194582 20789 raft_consensus.cc:2454] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e21d076c394e46ee96c4aeddc9cd4c97 in term 1.
I20260807 18:25:33.195106 21072 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97; no voters:
I20260807 18:25:33.195514 21276 raft_consensus.cc:2781] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:25:33.196096 21276 raft_consensus.cc:686] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 LEADER]: Becoming Leader. State: Replica: e21d076c394e46ee96c4aeddc9cd4c97, State: Running, Role: LEADER
I20260807 18:25:33.196655 21276 consensus_queue.cc:237] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:33.201792 20580 catalog_manager.cc:5495] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 reported cstate change: term changed from 0 to 1, leader changed from <none> to e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } health_report { overall_health: UNKNOWN } } }
I20260807 18:25:33.241082 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.235291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.29.156.195:45787)
0807 18:25:33.235516 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.235550 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.235653 (+ 103us) server_negotiation.cc:407] Connection header received
0807 18:25:33.235924 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.235928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.236039 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.236211 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:25:33.236602 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.237337 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.238785 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.239195 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.239334 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.239964 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:25:33.240463 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:25:33.240642 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.240695 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.240777 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:25:33.250880 20789 raft_consensus.cc:1261] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Refusing update from remote peer e21d076c394e46ee96c4aeddc9cd4c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:25:33.250880 20979 raft_consensus.cc:1261] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Refusing update from remote peer e21d076c394e46ee96c4aeddc9cd4c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:25:33.252358 21276 consensus_queue.cc:1035] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:25:33.252764 21280 consensus_queue.cc:1035] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:25:33.280083 20577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:25:33.280401 20577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:25:33.284379 20577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Sending DeleteTablet for 3 replicas of tablet 432b26b59f48443683fbd805811e36be
I20260807 18:25:33.285919 21147 tablet_service.cc:1514] Processing DeleteTablet for tablet 432b26b59f48443683fbd805811e36be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:60304
I20260807 18:25:33.285912 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet 432b26b59f48443683fbd805811e36be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:52128
I20260807 18:25:33.286659 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 20638
I20260807 18:25:33.286891 20950 tablet_service.cc:1514] Processing DeleteTablet for tablet 432b26b59f48443683fbd805811e36be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:25:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:53110
I20260807 18:25:33.289135 21301 tablet_replica.cc:330] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: stopping tablet replica
I20260807 18:25:33.289620 21301 raft_consensus.cc:2229] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:25:33.290331 21301 raft_consensus.cc:2258] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:33.292851 21300 tablet_replica.cc:330] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: stopping tablet replica
I20260807 18:25:33.293210 21303 tablet_replica.cc:330] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: stopping tablet replica
I20260807 18:25:33.293365 21300 raft_consensus.cc:2229] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:25:33.293767 21303 raft_consensus.cc:2229] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:25:33.294395 21303 raft_consensus.cc:2258] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:33.294598 21301 ts_tablet_manager.cc:1905] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:33.294983 21300 raft_consensus.cc:2258] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:25:33.296875 21303 ts_tablet_manager.cc:1905] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:33.303349 21303 ts_tablet_manager.cc:1918] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:25:33.303646 21303 log.cc:1196] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/432b26b59f48443683fbd805811e36be
I20260807 18:25:33.304415 21303 ts_tablet_manager.cc:1939] T 432b26b59f48443683fbd805811e36be P adaf21db71314f38b9e1481a95b97a58: Deleting consensus metadata
I20260807 18:25:33.304562 21300 ts_tablet_manager.cc:1905] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:25:33.306742 20559 catalog_manager.cc:4841] TS adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): tablet 432b26b59f48443683fbd805811e36be (table test-table [id=6532de5dd9274a789532e00e9ce68fa5]) successfully deleted
I20260807 18:25:33.307724 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.287976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.29.156.193:38015)
0807 18:25:33.288178 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.288184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.288202 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:25:33.288261 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.288266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.288345 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.288454 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:33.290440 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.291728 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.293809 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.294454 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.294614 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.298705 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:33.298721 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:33.298738 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:33.298788 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:33.302825 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.304075 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.304084 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.304088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.304268 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.305764 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.305770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.305773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.306274 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:33.306604 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.307057 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.307313 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:25:33.308561 20849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:33.312599 21301 ts_tablet_manager.cc:1918] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:25:33.312837 21301 log.cc:1196] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/432b26b59f48443683fbd805811e36be
I20260807 18:25:33.313418 21301 ts_tablet_manager.cc:1939] T 432b26b59f48443683fbd805811e36be P e21d076c394e46ee96c4aeddc9cd4c97: Deleting consensus metadata
I20260807 18:25:33.315258 20560 catalog_manager.cc:4841] TS e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): tablet 432b26b59f48443683fbd805811e36be (table test-table [id=6532de5dd9274a789532e00e9ce68fa5]) successfully deleted
I20260807 18:25:33.463305 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 20638
W20260807 18:25:33.491758 20562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.156.193:38015 (error 108)
I20260807 18:25:33.491914 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 20860
I20260807 18:25:33.495643 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.495069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52144)
0807 18:25:33.495252 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.495369 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:25:33.495965 20562 catalog_manager.cc:4584] TS 95f5894b2402434ead123b469c69a85e (127.29.156.193:38015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 432b26b59f48443683fbd805811e36be: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
I20260807 18:25:33.509456 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.492891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.29.156.194:39501)
0807 18:25:33.493099 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.493106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.493129 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:25:33.493258 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.493264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.493352 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.493476 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:33.494925 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.496152 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.498019 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.498473 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.498581 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.502144 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:33.502162 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:33.502179 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:33.502221 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:33.505908 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.506892 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.506901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.506904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.507084 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.507747 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.507752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.507754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.508190 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:33.508444 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.508831 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.509079 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:25:33.510418 21040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:33.549938 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.549331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52146)
0807 18:25:33.549582 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.549726 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:25:33.551185 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.550758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52148)
0807 18:25:33.550924 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.551040 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 18:25:33.610009 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 20860
I20260807 18:25:33.615949 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.615360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52158)
0807 18:25:33.615618 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.615728 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:25:33.617309 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.616815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52172)
0807 18:25:33.617044 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.617141 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 18:25:33.638926 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 21049
I20260807 18:25:33.657369 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.640101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.29.156.195:45787)
0807 18:25:33.640344 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:33.640350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:33.640374 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:25:33.640482 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:33.640488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:33.640585 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:33.640760 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:33.642455 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.643956 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.645509 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:33.646297 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:33.646452 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:33.649270 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:33.649294 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:33.649325 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:33.649376 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:33.653898 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.654814 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.654824 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.654828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.655032 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:33.655496 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:33.655504 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:33.655509 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:33.656078 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:33.656449 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:33.656627 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 18:25:33.656891 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:25:33.658303 21231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:33.698138 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.697511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52184)
0807 18:25:33.697779 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.697889 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 18:25:33.699360 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.698950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52188)
0807 18:25:33.699178 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.699249 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260807 18:25:33.800969 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 21049
I20260807 18:25:33.831883 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 20527
I20260807 18:25:33.833328 20628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:25:33.953282 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:33.952667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38015 (local address 127.0.0.1:52196)
0807 18:25:33.952883 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:33.953002 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:25:33.953282 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 20527
I20260807 18:25:53.980760 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--webserver_interface=127.29.156.254
--webserver_port=46563
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:25:54.342520 21311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:54.342875 21311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:54.343067 21311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:54.350378 21311 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:25:54.350536 21311 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:25:54.350611 21311 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:25:54.350689 21311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:25:54.350772 21311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:54.350875 21311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:54.350960 21311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:25:54.351042 21311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:25:54.361173 21311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=46563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:54.362550 21311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:54.363991 21311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:54.377369 21318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:54.377414 21319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:54.377293 21321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:54.377650 21311 server_base.cc:1034] running on GCE node
I20260807 18:25:54.378913 21311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:54.380643 21311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:54.381910 21311 hybrid_clock.cc:648] HybridClock initialized: now 1786127154381833 us; error 63 us; skew 500 ppm
I20260807 18:25:54.382323 21311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:54 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:54.390048 21311 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:25:54.391786 21311 webserver.cc:460] Webserver started at http://127.29.156.254:46563/ using document root <none> and password file <none>
I20260807 18:25:54.392586 21311 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:54.392729 21311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:54.398531 21311 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:25:54.401597 21332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:54.402971 21311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:25:54.403208 21311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "cdd7031236594757b48dcb1fa09a9442"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "63588b8c50ddde6abf1edf1239eba542"
server_key_iv: "243fd0af60eaafadebb73f415285ca12"
server_key_version: "encryptionkey@0"
I20260807 18:25:54.404277 21311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:54.431082 21311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:54.435864 21311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:54.436204 21311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:54.454918 21311 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44025
I20260807 18:25:54.454970 21405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44025 every 8 connection(s)
I20260807 18:25:54.456547 21311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:25:54.458053 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 21311
I20260807 18:25:54.458926 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:38015
--local_ip_for_outbound_sockets=127.29.156.193
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=43795
--webserver_interface=127.29.156.193
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:25:54.464025 21407 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:25:54.467386 21407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap starting.
I20260807 18:25:54.497174 21407 log.cc:824] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:54.517408 21407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:25:54.518021 21407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Bootstrap complete.
I20260807 18:25:54.527716 21407 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:54.528717 21407 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Initialized, Role: FOLLOWER
I20260807 18:25:54.529429 21407 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:54.529687 21407 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:25:54.529816 21407 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:25:54.530006 21407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:25:54.533973 21407 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:54.534487 21407 leader_election.cc:302] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdd7031236594757b48dcb1fa09a9442; no voters:
I20260807 18:25:54.535200 21407 leader_election.cc:288] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:25:54.535494 21411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:25:54.536737 21411 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [term 3 LEADER]: Becoming Leader. State: Replica: cdd7031236594757b48dcb1fa09a9442, State: Running, Role: LEADER
I20260807 18:25:54.537479 21407 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:25:54.537395 21411 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } }
I20260807 18:25:54.541556 21414 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd7031236594757b48dcb1fa09a9442. Latest consensus state: current_term: 3 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:54.541951 21414 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:54.546348 21413 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cdd7031236594757b48dcb1fa09a9442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd7031236594757b48dcb1fa09a9442" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44025 } } }
I20260807 18:25:54.546700 21413 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442 [sys.catalog]: This master's current role is: LEADER
I20260807 18:25:54.553963 21422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:25:54.564417 21422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:25:54.566218 21422 catalog_manager.cc:1259] Loaded cluster ID: 775ac5c0b9fd444f9b8a2cbc64907e7d
I20260807 18:25:54.566423 21422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:25:54.572250 21422 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:25:54.574395 21422 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Loaded TSK: 0
I20260807 18:25:54.574581 21422 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cdd7031236594757b48dcb1fa09a9442: Loaded TSK: 1
F20260807 18:25:54.575915 21422 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f69a33829ad google::LogMessage::Fail() at ??:0
@ 0x7f69a338588a google::LogMessage::SendToLog() at ??:0
@ 0x7f69a33824de google::LogMessage::Flush() at ??:0
@ 0x7f69a3383a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f69a70e67d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f69a70d573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f69b8db11c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f69b8db0783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f69b8e3c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f69b8e3c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f69b8e865e9 std::function<>::operator()() at ??:0
@ 0x7f69b8db4cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f69b8db2c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f69b8e3a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f69b8e3a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f69a5fce345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f69a5fd191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f69a5fd1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f69a5faceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f69a92866db start_thread at ??:0
@ 0x7f69a221c61f clone at ??:0
W20260807 18:25:54.897019 21409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:54.897360 21409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:54.897532 21409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:54.905344 21409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:54.905515 21409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:54.905700 21409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:25:54.915961 21409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:38015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=43795
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:54.917342 21409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:54.918695 21409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:54.934929 21474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:54.934907 21477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:54.935134 21409 server_base.cc:1034] running on GCE node
W20260807 18:25:54.934929 21475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:54.936278 21409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:54.937271 21409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:54.938486 21409 hybrid_clock.cc:648] HybridClock initialized: now 1786127154938414 us; error 61 us; skew 500 ppm
I20260807 18:25:54.938822 21409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:54 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:54.942476 21409 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:25:54.944269 21409 webserver.cc:460] Webserver started at http://127.29.156.193:43795/ using document root <none> and password file <none>
I20260807 18:25:54.945005 21409 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:54.945161 21409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:54.950681 21409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:25:54.953945 21488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:54.955286 21409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:25:54.955468 21409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "95f5894b2402434ead123b469c69a85e"
format_stamp: "Formatted at 2026-08-07 18:25:08 on dist-test-slave-sb2z"
server_key: "bcdad0ed977377cf1062bcee3b2b9266"
server_key_iv: "8c4e6032242a3085146f8511d5dc898e"
server_key_version: "encryptionkey@0"
I20260807 18:25:54.956506 21409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:54.984730 21409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:54.989791 21409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:54.990226 21409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:54.991360 21409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:54.993599 21497 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 18:25:55.000190 21409 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:25:55.000434 21409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 18:25:55.000622 21409 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:25:55.004513 21409 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:25:55.004657 21409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260807 18:25:55.005136 21497 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Bootstrap starting.
I20260807 18:25:55.015020 21497 log.cc:824] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:25:55.037581 21497 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:25:55.038645 21497 tablet_bootstrap.cc:492] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Bootstrap complete.
I20260807 18:25:55.039435 21497 ts_tablet_manager.cc:1397] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Time spent bootstrapping tablet: real 0.035s user 0.030s sys 0.000s
I20260807 18:25:55.050652 21409 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38015
I20260807 18:25:55.050676 21674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38015 every 8 connection(s)
I20260807 18:25:55.052703 21409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:25:55.053530 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 21409
I20260807 18:25:55.054630 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:39501
--local_ip_for_outbound_sockets=127.29.156.194
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=35959
--webserver_interface=127.29.156.194
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:25:55.054502 21497 raft_consensus.cc:348] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:55.056727 21497 raft_consensus.cc:729] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f5894b2402434ead123b469c69a85e, State: Initialized, Role: FOLLOWER
I20260807 18:25:55.057915 21497 consensus_queue.cc:260] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:55.062428 21497 ts_tablet_manager.cc:1428] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e: Time spent starting tablet: real 0.023s user 0.017s sys 0.004s
I20260807 18:25:55.065450 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.056325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:60921)
0807 18:25:55.064173 (+ 7848us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.064484 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7513,"thread_start_us":7270,"threads_started":1}
W20260807 18:25:55.066263 21676 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:44025 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:44025: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
I20260807 18:25:55.067286 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.066910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:52859)
0807 18:25:55.067081 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.067178 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:25:55.068377 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.068017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:46841)
0807 18:25:55.068175 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.068252 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:25:55.068701 21676 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:25:55.426450 21680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:55.426769 21680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:55.426869 21680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:55.434274 21680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:55.434391 21680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:55.434505 21680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:25:55.444448 21680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:39501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=35959
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:55.445787 21680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:55.447134 21680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:55.461581 21686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:55.461651 21687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:55.461895 21693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:55.462776 21680 server_base.cc:1034] running on GCE node
I20260807 18:25:55.463512 21680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:55.464622 21680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:55.465884 21680 hybrid_clock.cc:648] HybridClock initialized: now 1786127155465815 us; error 60 us; skew 500 ppm
I20260807 18:25:55.466235 21680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:55 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127155, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:55.469683 21680 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:25:55.471380 21680 webserver.cc:460] Webserver started at http://127.29.156.194:35959/ using document root <none> and password file <none>
I20260807 18:25:55.472147 21680 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:55.472306 21680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:55.477802 21680 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:25:55.480782 21706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:55.482200 21680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:25:55.482398 21680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "adaf21db71314f38b9e1481a95b97a58"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "30a863b4fe39a5356be6a9dbcb7cb358"
server_key_iv: "42c18e063eddc42cbdbd163358090154"
server_key_version: "encryptionkey@0"
I20260807 18:25:55.483439 21680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:55.503185 21680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:55.507750 21680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:55.508152 21680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:55.509595 21680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:55.512070 21680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:55.512209 21680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:55.512329 21680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:55.512429 21680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:55.549190 21680 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:39501
I20260807 18:25:55.549284 21877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:39501 every 8 connection(s)
I20260807 18:25:55.550661 21680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:25:55.552066 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 21680
I20260807 18:25:55.552996 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:45787
--local_ip_for_outbound_sockets=127.29.156.195
--tserver_master_addrs=127.29.156.254:44025
--webserver_port=34439
--webserver_interface=127.29.156.195
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:25:55.556720 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.554159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:49739)
0807 18:25:55.555162 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.555479 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":345,"threads_started":1}
W20260807 18:25:55.557588 21878 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:44025 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:44025: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
I20260807 18:25:55.558657 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.558174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:34761)
0807 18:25:55.558376 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.558479 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:25:55.559746 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:55.559256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:39265)
0807 18:25:55.559437 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:55.559582 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:25:55.559998 21878 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:25:55.908715 21883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:25:55.909001 21883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:25:55.909175 21883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:25:55.916671 21883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:25:55.916790 21883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:25:55.917013 21883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:25:55.927068 21883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:45787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=34439
--tserver_master_addrs=127.29.156.254:44025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:25:55.928666 21883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:25:55.930114 21883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:25:55.944408 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:55.944487 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:25:55.944449 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:25:55.945572 21883 server_base.cc:1034] running on GCE node
I20260807 18:25:55.946244 21883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:25:55.947239 21883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:25:55.948485 21883 hybrid_clock.cc:648] HybridClock initialized: now 1786127155948419 us; error 77 us; skew 500 ppm
I20260807 18:25:55.948836 21883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:25:55 dist-test-slave-sb2z krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127155, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:25:55.952243 21883 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:25:55.953704 21883 webserver.cc:460] Webserver started at http://127.29.156.195:34439/ using document root <none> and password file <none>
I20260807 18:25:55.954389 21883 fs_manager.cc:359] Metadata directory not provided
I20260807 18:25:55.954496 21883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:25:55.960319 21883 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:25:55.963073 21902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:55.964686 21883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:25:55.964855 21883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "e21d076c394e46ee96c4aeddc9cd4c97"
format_stamp: "Formatted at 2026-08-07 18:25:09 on dist-test-slave-sb2z"
server_key: "d96fa2036a6e654c87b6ab11a63c41f2"
server_key_iv: "c344f5bf05a3eb4cb382614bbdd8e740"
server_key_version: "encryptionkey@0"
I20260807 18:25:55.965931 21883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:25:55.991379 21883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:25:55.996323 21883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:25:55.996650 21883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:25:55.997987 21883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:25:55.999481 21883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:25:55.999657 21883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:25:55.999784 21883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:25:55.999856 21883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:25:56.037684 21883 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:45787
I20260807 18:25:56.037756 22065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:45787 every 8 connection(s)
I20260807 18:25:56.039222 21883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:25:56.044516 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.042473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:43567)
0807 18:25:56.043260 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.043591 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":293,"threads_started":1}
W20260807 18:25:56.045219 22069 heartbeater.cc:644] Failed to heartbeat to 127.29.156.254:44025 (0 consecutive failures): Network error: Failed to ping master at 127.29.156.254:44025: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
I20260807 18:25:56.046056 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.045736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:45085)
0807 18:25:56.045876 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.045946 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 18:25:56.046967 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.046631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:55833)
0807 18:25:56.046772 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.046858 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:25:56.047214 22069 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:25:56.047982 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 21883
I20260807 18:25:56.070197 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.069375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:49217)
0807 18:25:56.069876 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.069962 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":228,"threads_started":1}
I20260807 18:25:56.293226 22076 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:25:56.293568 22076 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:56.295354 22076 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787), adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
Aug 07 18:25:56 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
Aug 07 18:25:56 dist-test-slave-sb2z krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:25:56.322139 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.295689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:39501 (local address 127.29.156.193:39859)
0807 18:25:56.295968 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.296002 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:56.296433 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:56.300653 (+ 4220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:56.300661 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:56.301681 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:56.302968 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:56.302996 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.305498 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.305504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:56.308033 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:56.308050 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.309309 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.309314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:56.309495 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:56.310314 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:56.310358 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:56.313184 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:56.317901 (+ 4717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:56.317911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:56.317927 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:56.318788 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:56.319399 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:56.319403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:56.319405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:56.319702 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:56.320589 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:56.320595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:56.320869 (+ 274us) client_negotiation.cc:769] Sending connection context
0807 18:25:56.321168 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:25:56.321660 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":415,"spinlock_wait_cycles":25216}
I20260807 18:25:56.322243 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.296087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:39859 (local address 127.29.156.194:39501)
0807 18:25:56.297137 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:56.297148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:56.297189 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:25:56.299594 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:56.299607 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:56.300128 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:56.300530 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:56.303194 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.305269 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.308197 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.309095 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.309279 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:56.313490 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:56.313497 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:56.313512 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:56.313550 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:56.317725 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:56.318963 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:56.318972 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:56.318981 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:56.319228 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:56.319856 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:56.319860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:56.319862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:56.320442 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:56.320589 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:56.321415 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 18:25:56.321828 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":760,"threads_started":1}
I20260807 18:25:56.322408 22077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.295794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:45787 (local address 127.29.156.193:48075)
0807 18:25:56.297641 (+ 1847us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.297666 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:25:56.297873 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:25:56.300907 (+ 3034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:25:56.300917 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:25:56.301683 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:25:56.302968 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:56.302997 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.305503 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.305507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:56.308033 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:25:56.308050 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.309742 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.309749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:25:56.309940 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:56.310603 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:25:56.310620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:25:56.313512 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:25:56.317901 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:56.317911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:56.317927 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:56.318787 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:56.319357 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:25:56.319364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:25:56.319367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:25:56.319710 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:25:56.321306 (+ 1596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:25:56.321310 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:25:56.321461 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 18:25:56.321623 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:25:56.321897 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1698,"mutex_wait_us":303,"spinlock_wait_cycles":18816,"thread_start_us":1514,"threads_started":1}
I20260807 18:25:56.322726 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.296301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:48075 (local address 127.29.156.195:45787)
0807 18:25:56.297125 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 18:25:56.297142 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:25:56.297761 (+ 619us) server_negotiation.cc:407] Connection header received
0807 18:25:56.300148 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:25:56.300161 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:25:56.300577 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:25:56.300930 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:25:56.303194 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.305270 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.308550 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:25:56.309523 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:25:56.309742 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:25:56.313808 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:25:56.313818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:25:56.313837 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:25:56.313881 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:25:56.317735 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:56.318962 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:56.318972 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:56.318981 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:56.319201 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:25:56.320653 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:25:56.320656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:25:56.320658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:25:56.321156 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:25:56.321365 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:25:56.321891 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 18:25:56.322397 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":411,"threads_started":1}
W20260807 18:25:56.325898 21490 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
W20260807 18:25:56.326184 21492 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
I20260807 18:25:56.326355 21490 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f5894b2402434ead123b469c69a85e; no voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97
I20260807 18:25:56.326814 22076 raft_consensus.cc:2726] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:25:56.561533 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:56.560583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:51303)
0807 18:25:56.561138 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:56.561289 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":298,"threads_started":1}
I20260807 18:25:57.048617 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:57.047795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:43651)
0807 18:25:57.048257 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:57.048390 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":232,"threads_started":1}
I20260807 18:25:57.072157 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:57.071357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:36449)
0807 18:25:57.071904 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:57.071991 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":325,"threads_started":1}
I20260807 18:25:57.563223 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:57.562487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:59987)
0807 18:25:57.562958 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:57.563046 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":258,"threads_started":1}
I20260807 18:25:58.050525 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:58.049626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:60059)
0807 18:25:58.050242 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:58.050327 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":361,"threads_started":1}
I20260807 18:25:58.074080 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:58.073137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:56731)
0807 18:25:58.073739 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:58.073829 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":275,"threads_started":1}
I20260807 18:25:58.461522 22087 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:25:58.461777 22087 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:25:58.462558 22087 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787), adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
W20260807 18:25:58.463639 21490 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
W20260807 18:25:58.463816 21492 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
I20260807 18:25:58.463984 21492 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f5894b2402434ead123b469c69a85e; no voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97
I20260807 18:25:58.464249 22087 raft_consensus.cc:2726] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:25:58.565162 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:58.564251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:39335)
0807 18:25:58.564895 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:58.564985 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":427,"threads_started":1}
I20260807 18:25:59.052381 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:59.051511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:42291)
0807 18:25:59.052106 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:59.052202 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":296,"threads_started":1}
I20260807 18:25:59.075946 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:59.075101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:46463)
0807 18:25:59.075668 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:59.075758 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":316,"threads_started":1}
I20260807 18:25:59.566919 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:25:59.566175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:55117)
0807 18:25:59.566653 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 18:25:59.566737 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260807 18:26:00.054394 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:00.053466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:41303)
0807 18:26:00.054098 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:00.054204 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":383,"threads_started":1}
I20260807 18:26:00.078004 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:00.077101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:44717)
0807 18:26:00.077698 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:00.077791 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":322,"threads_started":1}
I20260807 18:26:00.568879 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:00.568100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:41887)
0807 18:26:00.568614 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:00.568706 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":291,"threads_started":1}
I20260807 18:26:01.056385 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:01.055440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:37975)
0807 18:26:01.056044 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:01.056188 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":388,"threads_started":1}
I20260807 18:26:01.079950 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:01.079115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:55697)
0807 18:26:01.079660 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:01.079767 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":256,"threads_started":1}
I20260807 18:26:01.321774 22100 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:01.322090 22100 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:26:01.323029 22100 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787), adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
W20260807 18:26:01.324515 21490 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
W20260807 18:26:01.324805 21492 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
I20260807 18:26:01.324990 21492 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f5894b2402434ead123b469c69a85e; no voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97
I20260807 18:26:01.325270 22100 raft_consensus.cc:2726] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:26:01.570966 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:01.569973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:40373)
0807 18:26:01.570636 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:01.570727 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":418,"threads_started":1}
I20260807 18:26:02.058329 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:02.057453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:56553)
0807 18:26:02.058045 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:02.058134 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":340,"threads_started":1}
I20260807 18:26:02.081852 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:02.081011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:57793)
0807 18:26:02.081547 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:02.081643 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":295,"threads_started":1}
I20260807 18:26:02.573350 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:02.572261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:44811)
0807 18:26:02.572967 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:02.573077 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":361,"threads_started":1}
I20260807 18:26:03.060387 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:03.059373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:39127)
0807 18:26:03.059979 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:03.060090 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":253,"threads_started":1}
I20260807 18:26:03.083976 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:03.083105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:52243)
0807 18:26:03.083713 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:03.083804 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":321,"threads_started":1}
I20260807 18:26:03.575825 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:03.574581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:53003)
0807 18:26:03.575346 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:03.575473 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":437,"threads_started":1}
I20260807 18:26:04.062245 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:04.061508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:36137)
0807 18:26:04.061942 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:04.062032 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":218,"threads_started":1}
I20260807 18:26:04.086004 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:04.085123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:45587)
0807 18:26:04.085667 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:04.085783 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":280,"threads_started":1}
I20260807 18:26:04.578006 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:04.577017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:37635)
0807 18:26:04.577690 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:04.577792 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":331,"threads_started":1}
I20260807 18:26:05.064440 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:05.063442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:53863)
0807 18:26:05.064118 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:05.064236 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":395,"threads_started":1}
I20260807 18:26:05.088397 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:05.087422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:44083)
0807 18:26:05.088101 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:05.088198 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":401,"threads_started":1}
I20260807 18:26:05.580117 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:05.579207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:42807)
0807 18:26:05.579818 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:05.579908 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":385,"threads_started":1}
I20260807 18:26:05.969322 22118 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:05.969696 22118 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:26:05.970441 22118 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787), adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
W20260807 18:26:05.971807 21490 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
W20260807 18:26:05.972050 21492 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
I20260807 18:26:05.972232 21492 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f5894b2402434ead123b469c69a85e; no voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97
I20260807 18:26:05.972523 22118 raft_consensus.cc:2726] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:26:06.066315 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:06.065549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:57539)
0807 18:26:06.066046 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:06.066134 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":269,"threads_started":1}
I20260807 18:26:06.090514 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:06.089549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:53923)
0807 18:26:06.090186 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:06.090307 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":285,"threads_started":1}
I20260807 18:26:06.582151 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:06.581396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:44661)
0807 18:26:06.581879 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:06.581969 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":230,"threads_started":1}
I20260807 18:26:07.068222 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:07.067331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:37175)
0807 18:26:07.067939 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:07.068031 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":348,"threads_started":1}
I20260807 18:26:07.092516 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:07.091691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:39077)
0807 18:26:07.092247 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:07.092339 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":322,"threads_started":1}
I20260807 18:26:07.584220 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:07.583265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:35531)
0807 18:26:07.583934 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:07.584027 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":340,"threads_started":1}
I20260807 18:26:08.006333 22132 raft_consensus.cc:482] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:08.006538 22132 raft_consensus.cc:504] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21d076c394e46ee96c4aeddc9cd4c97" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 45787 } } peers { permanent_uuid: "adaf21db71314f38b9e1481a95b97a58" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 39501 } } peers { permanent_uuid: "95f5894b2402434ead123b469c69a85e" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38015 } }
I20260807 18:26:08.007186 22132 leader_election.cc:288] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787), adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501)
W20260807 18:26:08.008510 21492 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e21d076c394e46ee96c4aeddc9cd4c97 (127.29.156.195:45787): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
W20260807 18:26:08.008687 21490 leader_election.cc:341] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer adaf21db71314f38b9e1481a95b97a58 (127.29.156.194:39501): Not found: Tablet not found: 432b26b59f48443683fbd805811e36be
I20260807 18:26:08.008858 21490 leader_election.cc:302] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f5894b2402434ead123b469c69a85e; no voters: adaf21db71314f38b9e1481a95b97a58, e21d076c394e46ee96c4aeddc9cd4c97
I20260807 18:26:08.009179 22132 raft_consensus.cc:2726] T 432b26b59f48443683fbd805811e36be P 95f5894b2402434ead123b469c69a85e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:26:08.070366 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:08.069508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:58017)
0807 18:26:08.070087 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:08.070184 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":278,"threads_started":1}
I20260807 18:26:08.094439 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:08.093521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:58345)
0807 18:26:08.094047 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:08.094190 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":253,"threads_started":1}
I20260807 18:26:08.586382 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:08.585274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:60999)
0807 18:26:08.585972 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:08.586179 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":468,"threads_started":1}
I20260807 18:26:09.072302 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:09.071410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:58341)
0807 18:26:09.072031 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:09.072134 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":367,"threads_started":1}
I20260807 18:26:09.096532 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:09.095711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:60277)
0807 18:26:09.096218 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:09.096312 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":247,"threads_started":1}
I20260807 18:26:09.588586 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:09.587429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:52741)
0807 18:26:09.588165 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:09.588299 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":321,"threads_started":1}
I20260807 18:26:10.074251 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:10.073426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:60531)
0807 18:26:10.073936 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:10.074036 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260807 18:26:10.098631 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:10.097846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:37441)
0807 18:26:10.098321 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:10.098416 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":244,"threads_started":1}
I20260807 18:26:10.590500 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:10.589726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:42637)
0807 18:26:10.590212 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:10.590340 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":244,"threads_started":1}
I20260807 18:26:11.061400 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 21409
I20260807 18:26:11.076568 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.075611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.195:48479)
0807 18:26:11.076250 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:11.076382 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":312,"threads_started":1}
I20260807 18:26:11.100828 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.099960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.193:59235)
0807 18:26:11.100524 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:11.100634 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":268,"threads_started":1}
I20260807 18:26:11.592694 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.591790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44025 (local address 127.29.156.194:42163)
0807 18:26:11.592400 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:11.592500 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.254:44025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":320,"threads_started":1}
I20260807 18:26:11.685711 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.062823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.29.156.193:38015)
0807 18:26:11.063581 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:11.063595 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:11.672287 (+608692us) server_negotiation.cc:407] Connection header received
0807 18:26:11.672409 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:11.672419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:11.672677 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:11.672834 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:11.674336 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.675303 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.676619 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.677228 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.677329 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:11.679981 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:11.679993 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:11.680008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:11.680050 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:11.682918 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.683756 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.683764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.683767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.683917 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.684285 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.684288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.684290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.684652 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:11.684879 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:11.685209 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:26:11.685423 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":574,"threads_started":1}
I20260807 18:26:11.687377 21664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:11.779138 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 21409
I20260807 18:26:11.807083 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 21680
I20260807 18:26:11.821830 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.808172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.29.156.194:39501)
0807 18:26:11.808844 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:11.808848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:11.808866 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:26:11.808922 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:11.808927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:11.808985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:11.809081 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:11.810816 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.811749 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.813011 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.813472 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.813575 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:11.816424 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:11.816431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:11.816434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:11.816488 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:11.819326 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.820107 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.820112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.820114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.820279 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.820611 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.820615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.820617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.820879 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:11.821143 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:11.821336 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 18:26:11.821518 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":402,"threads_started":1}
I20260807 18:26:11.822566 21873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:11.908103 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 21680
I20260807 18:26:11.932480 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 21883
I20260807 18:26:11.947875 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:11.933406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.29.156.195:45787)
0807 18:26:11.933958 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:11.933963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:11.933981 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:26:11.934039 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:11.934042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:11.934103 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:11.934204 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:11.935814 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.936634 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.937926 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:11.938357 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:11.938455 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:11.941414 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:11.941419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:11.941423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:11.941461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:11.944903 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.945672 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.945676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.945679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.945836 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:11.946225 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:11.946229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:11.946231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:11.946526 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:11.946785 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:11.947168 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:26:11.947361 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":310,"threads_started":1}
I20260807 18:26:11.948674 22056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:12.035921 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 21883
2026-08-07T18:26:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66016 ms)
[----------] 30 tests from SecurityITest (323412 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:12 dist-test-slave-sb2z krb5kdc[22159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:12 dist-test-slave-sb2z krb5kdc[22159](info): set up 2 sockets
Aug 07 18:26:12 dist-test-slave-sb2z krb5kdc[22159](info): commencing operation
krb5kdc: starting...
W20260807 18:26:13.392323 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.002s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:13 dist-test-slave-sb2z krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:26:13.740198 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44997
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:14.115253 22176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:14.115643 22176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:14.115758 22176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:14.123080 22176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:14.123193 22176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:14.123246 22176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:14.123306 22176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:14.123351 22176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:14.133487 22176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:14.134891 22176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:14.136215 22176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:14.149006 22185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:14.149042 22182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:14.149008 22188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:14.149842 22176 server_base.cc:1034] running on GCE node
I20260807 18:26:14.150806 22176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:14.152403 22176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:14.153645 22176 hybrid_clock.cc:648] HybridClock initialized: now 1786127174153584 us; error 34 us; skew 500 ppm
I20260807 18:26:14.154052 22176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:14 dist-test-slave-sb2z krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127174, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:14.157544 22176 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:26:14.159153 22176 webserver.cc:460] Webserver started at http://127.29.156.254:42579/ using document root <none> and password file <none>
I20260807 18:26:14.159930 22176 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:14.160043 22176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:14.160312 22176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:14.163461 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "2bcdf89e3997c5761cb7f774ff2b0e16"
server_key_iv: "bc1ee484d825f6833aad0d45c5e9a14c"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.164268 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "2bcdf89e3997c5761cb7f774ff2b0e16"
server_key_iv: "bc1ee484d825f6833aad0d45c5e9a14c"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.169724 22176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:26:14.173449 22195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:14.174955 22176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:26:14.175149 22176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "2bcdf89e3997c5761cb7f774ff2b0e16"
server_key_iv: "bc1ee484d825f6833aad0d45c5e9a14c"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.175345 22176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:14.202239 22176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:14.207055 22176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:14.207315 22176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:14.228215 22176 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44997
I20260807 18:26:14.228185 22277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44997 every 8 connection(s)
I20260807 18:26:14.229754 22176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:14.233846 22278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:14.236491 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 22176
I20260807 18:26:14.236747 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:14.237452 30323 external_mini_cluster.cc:1428] Setting key 01e7d2b413bdef5c369ddd5ed501243c
I20260807 18:26:14.247686 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: Bootstrap starting.
I20260807 18:26:14.251793 22278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:14.252895 22278 log.cc:824] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:26:14 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:14.256426 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: No bootstrap required, opened a new log
I20260807 18:26:14.264304 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:14.239837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.29.156.254:44997)
0807 18:26:14.240680 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:14.240696 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:14.240750 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:26:14.243264 (+ 2514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:14.243299 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:14.243920 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:14.244403 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:14.245826 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.247895 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.249392 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.250526 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.250741 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:14.255654 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:14.255666 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:14.255687 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:14.255739 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:14.259368 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:14.260495 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:14.260502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:14.260509 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:14.260724 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:14.261224 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:14.261227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:14.261230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:14.261855 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:14.262083 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:14.262585 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 18:26:14.263017 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":356,"threads_started":1}
I20260807 18:26:14.265646 22278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } }
I20260807 18:26:14.266145 22278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:14.266273 22278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991b8fd7aa8b4c1f8b44b7d0abb166fb, State: Initialized, Role: FOLLOWER
I20260807 18:26:14.266943 22278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } }
I20260807 18:26:14.267242 22278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:14.267431 22278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:14.267680 22278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:14.269927 22278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } }
I20260807 18:26:14.270381 22278 leader_election.cc:302] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 991b8fd7aa8b4c1f8b44b7d0abb166fb; no voters:
I20260807 18:26:14.271008 22278 leader_election.cc:288] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:14.271199 22286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:14.272791 22286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [term 1 LEADER]: Becoming Leader. State: Replica: 991b8fd7aa8b4c1f8b44b7d0abb166fb, State: Running, Role: LEADER
I20260807 18:26:14.273638 22278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:14.273556 22286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } }
I20260807 18:26:14.277150 22288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 991b8fd7aa8b4c1f8b44b7d0abb166fb. Latest consensus state: current_term: 1 leader_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } } }
I20260807 18:26:14.277606 22288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:14.277851 22287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "991b8fd7aa8b4c1f8b44b7d0abb166fb" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44997 } } }
I20260807 18:26:14.278213 22287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:14.280367 22295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:14.285758 22295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:14.295553 22295 catalog_manager.cc:1347] Generated new cluster ID: 6875b5491ab24dfbb8253a55be9cdd4c
I20260807 18:26:14.295681 22295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:14.330986 22295 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:14.333069 22295 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:14.342620 22295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: Generated new TSK 0
I20260807 18:26:14.343703 22295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:26:14.435899 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:14.797243 22318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:14.797597 22318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:14.797770 22318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:14.805429 22318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:14.805608 22318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:14.805788 22318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:14.816187 22318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:14.817600 22318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:14.818943 22318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:14.832676 22328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:14.832674 22329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:14.832657 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:14.833781 22318 server_base.cc:1034] running on GCE node
I20260807 18:26:14.834414 22318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:14.835352 22318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:14.836572 22318 hybrid_clock.cc:648] HybridClock initialized: now 1786127174836556 us; error 73 us; skew 500 ppm
I20260807 18:26:14.836899 22318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:14 dist-test-slave-sb2z krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127174, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:14.840430 22318 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:26:14.842156 22318 webserver.cc:460] Webserver started at http://127.29.156.193:41389/ using document root <none> and password file <none>
I20260807 18:26:14.842959 22318 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:14.843113 22318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:14.843448 22318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:14.846513 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "3c7be0b4ef6d421db89d96c9dcac566c"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "9ec931de64e17f779c64d0f5ca043a30"
server_key_iv: "7c2ff307c2d948a56b9cbbbe85965fbc"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.847344 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "3c7be0b4ef6d421db89d96c9dcac566c"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "9ec931de64e17f779c64d0f5ca043a30"
server_key_iv: "7c2ff307c2d948a56b9cbbbe85965fbc"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.852977 22318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:26:14.856621 22343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:14.858007 22318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:14.858240 22318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "3c7be0b4ef6d421db89d96c9dcac566c"
format_stamp: "Formatted at 2026-08-07 18:26:14 on dist-test-slave-sb2z"
server_key: "9ec931de64e17f779c64d0f5ca043a30"
server_key_iv: "7c2ff307c2d948a56b9cbbbe85965fbc"
server_key_version: "encryptionkey@0"
I20260807 18:26:14.858453 22318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:14.873982 22318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:14.878347 22318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:14.878732 22318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:14.879959 22318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:14.881372 22318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:14.881505 22318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:14.881626 22318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:14.881677 22318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:14.917166 22318 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:39069
I20260807 18:26:14.917213 22511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:39069 every 8 connection(s)
I20260807 18:26:14.918543 22318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:14.922703 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 22318
I20260807 18:26:14.922948 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:14.923671 30323 external_mini_cluster.cc:1428] Setting key b4e31bf44ecb555db64efadfe02e101a
Aug 07 18:26:14 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127174, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:14.941973 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:14.921239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:34933 (local address 127.29.156.254:44997)
0807 18:26:14.921753 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:14.921758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:14.922601 (+ 843us) server_negotiation.cc:407] Connection header received
0807 18:26:14.924965 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:14.924969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:14.925052 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:14.925159 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:14.927007 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.928256 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.930022 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.930749 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.930902 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:14.934557 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:14.934566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:14.934579 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:14.934639 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:14.938483 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:14.939280 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:14.939289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:14.939295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:14.939454 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:14.940048 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:14.940055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:14.940061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:14.940470 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:14.940628 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:14.941516 (+ 888us) server_negotiation.cc:299] Negotiation successful
0807 18:26:14.941707 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":267,"threads_started":1}
I20260807 18:26:14.943667 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:14.921673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44997 (local address 127.29.156.193:34933)
0807 18:26:14.922411 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:14.922455 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:14.924602 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:14.925445 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:14.925457 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:14.926230 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:14.926818 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:14.926835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.928498 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.928502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:14.929845 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:14.929857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:14.930998 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:14.931003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:14.931237 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:14.931983 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:14.932013 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:14.934305 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:14.938649 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:14.938657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:14.938670 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:14.939143 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:14.939682 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:14.939687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:14.939689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:14.939900 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:14.940612 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:14.940618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:14.941027 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 18:26:14.941422 (+ 395us) client_negotiation.cc:240] Negotiation successful
0807 18:26:14.942072 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":283,"threads_started":1}
I20260807 18:26:14.945570 22512 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44997
I20260807 18:26:14.945915 22512 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:14.946699 22512 heartbeater.cc:511] Master 127.29.156.254:44997 requested a full tablet report, sending...
I20260807 18:26:14.949285 22223 ts_manager.cc:194] Registered new tserver with Master: 3c7be0b4ef6d421db89d96c9dcac566c (127.29.156.193:39069)
I20260807 18:26:14.951624 22223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:34933
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:26:15.008351 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:15.371398 22521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:15.371902 22521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:15.372087 22521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:15.385381 22521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:15.385558 22521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:15.385815 22521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:15.402611 22521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:15.404565 22521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:15.407869 22521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:15.425880 22527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:15.426548 22528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:15.427160 22533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:15.427978 22521 server_base.cc:1034] running on GCE node
I20260807 18:26:15.428740 22521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:15.429875 22521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:15.431135 22521 hybrid_clock.cc:648] HybridClock initialized: now 1786127175431057 us; error 57 us; skew 500 ppm
I20260807 18:26:15.431511 22521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:15 dist-test-slave-sb2z krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:15.435618 22521 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:26:15.438362 22521 webserver.cc:460] Webserver started at http://127.29.156.194:33153/ using document root <none> and password file <none>
I20260807 18:26:15.439141 22521 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:15.439303 22521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:15.439754 22521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:15.443040 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "aa4b175fedfb4f2bacf715574623d3f8"
format_stamp: "Formatted at 2026-08-07 18:26:15 on dist-test-slave-sb2z"
server_key: "40cfcd2a4ed919e7b975c83e9b6f3ef6"
server_key_iv: "100c95f22661d23ad40de06c472eb262"
server_key_version: "encryptionkey@0"
I20260807 18:26:15.443915 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "aa4b175fedfb4f2bacf715574623d3f8"
format_stamp: "Formatted at 2026-08-07 18:26:15 on dist-test-slave-sb2z"
server_key: "40cfcd2a4ed919e7b975c83e9b6f3ef6"
server_key_iv: "100c95f22661d23ad40de06c472eb262"
server_key_version: "encryptionkey@0"
I20260807 18:26:15.449352 22521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:26:15.453034 22544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:15.454444 22521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:15.454648 22521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "aa4b175fedfb4f2bacf715574623d3f8"
format_stamp: "Formatted at 2026-08-07 18:26:15 on dist-test-slave-sb2z"
server_key: "40cfcd2a4ed919e7b975c83e9b6f3ef6"
server_key_iv: "100c95f22661d23ad40de06c472eb262"
server_key_version: "encryptionkey@0"
I20260807 18:26:15.454876 22521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:15.471396 22521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:15.476608 22521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:15.476945 22521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:15.478125 22521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:15.479841 22521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:15.480017 22521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:15.480221 22521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:15.480278 22521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:15.516942 22521 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:46057
I20260807 18:26:15.516995 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:46057 every 8 connection(s)
I20260807 18:26:15.518388 22521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:15.519130 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 22521
I20260807 18:26:15.519395 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:15.520179 30323 external_mini_cluster.cc:1428] Setting key 6ae5e70064f333cd935fe214b14514dc
Aug 07 18:26:15 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127175, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:15.541091 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:15.521489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:35935 (local address 127.29.156.254:44997)
0807 18:26:15.521984 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:15.521991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:15.523155 (+ 1164us) server_negotiation.cc:407] Connection header received
0807 18:26:15.525281 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:15.525287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:15.525389 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:15.525536 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:15.527242 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:15.528135 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:15.529883 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:15.530364 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:15.530545 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:15.534529 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:15.534534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:15.534538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:15.534575 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:15.537604 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:15.538475 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:15.538479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:15.538482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:15.538626 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:15.539142 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:15.539148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:15.539151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:15.539442 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:15.539642 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:15.540426 (+ 784us) server_negotiation.cc:299] Negotiation successful
0807 18:26:15.540674 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260807 18:26:15.542176 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:15.522019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44997 (local address 127.29.156.194:35935)
0807 18:26:15.522968 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:15.523018 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:15.524978 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:15.525824 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:15.525833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:15.526410 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:15.527036 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:15.527056 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:15.528323 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:15.528328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:15.529710 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:15.529723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:15.530652 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:15.530660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:15.530914 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:15.531771 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:15.531797 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:15.534244 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:15.537792 (+ 3548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:15.537806 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:15.537829 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:15.538349 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:15.538770 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:15.538775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:15.538779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:15.539009 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:15.539631 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:15.539640 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:15.540011 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 18:26:15.540333 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 18:26:15.540941 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":386,"threads_started":1}
I20260807 18:26:15.544008 22703 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44997
I20260807 18:26:15.544345 22703 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:15.545123 22703 heartbeater.cc:511] Master 127.29.156.254:44997 requested a full tablet report, sending...
I20260807 18:26:15.546810 22223 ts_manager.cc:194] Registered new tserver with Master: aa4b175fedfb4f2bacf715574623d3f8 (127.29.156.194:46057)
I20260807 18:26:15.548286 22223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:35935
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:26:15.604071 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:26:15.954871 22512 heartbeater.cc:503] Master 127.29.156.254:44997 was elected leader, sending a full tablet report...
W20260807 18:26:15.975250 22710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:15.975689 22710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:15.975886 22710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:15.984853 22710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:15.985010 22710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:15.985245 22710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:15.996289 22710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:15.997852 22710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:15.999346 22710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:16.016079 22718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:16.016201 22724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:16.016276 22719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:16.017568 22710 server_base.cc:1034] running on GCE node
I20260807 18:26:16.018392 22710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:16.019682 22710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:16.020946 22710 hybrid_clock.cc:648] HybridClock initialized: now 1786127176020874 us; error 59 us; skew 500 ppm
I20260807 18:26:16.021343 22710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:16 dist-test-slave-sb2z krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:16.025461 22710 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:26:16.027244 22710 webserver.cc:460] Webserver started at http://127.29.156.195:35461/ using document root <none> and password file <none>
I20260807 18:26:16.028102 22710 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:16.028296 22710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:16.028645 22710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:16.031981 22710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "f6b5480504e94858aa937cd0345f3cbd"
format_stamp: "Formatted at 2026-08-07 18:26:16 on dist-test-slave-sb2z"
server_key: "f64b853b77e9a314b4781e22e14df057"
server_key_iv: "197826fb8a3961c14c901b9fcbdd0443"
server_key_version: "encryptionkey@0"
I20260807 18:26:16.032752 22710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "f6b5480504e94858aa937cd0345f3cbd"
format_stamp: "Formatted at 2026-08-07 18:26:16 on dist-test-slave-sb2z"
server_key: "f64b853b77e9a314b4781e22e14df057"
server_key_iv: "197826fb8a3961c14c901b9fcbdd0443"
server_key_version: "encryptionkey@0"
I20260807 18:26:16.038373 22710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:26:16.043226 22733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:16.044879 22710 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:26:16.045092 22710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "f6b5480504e94858aa937cd0345f3cbd"
format_stamp: "Formatted at 2026-08-07 18:26:16 on dist-test-slave-sb2z"
server_key: "f64b853b77e9a314b4781e22e14df057"
server_key_iv: "197826fb8a3961c14c901b9fcbdd0443"
server_key_version: "encryptionkey@0"
I20260807 18:26:16.045325 22710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:16.060415 22710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:16.065449 22710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:16.065783 22710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:16.066895 22710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:16.068315 22710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:16.068392 22710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:16.068528 22710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:16.068585 22710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:16.106215 22710 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:38573
I20260807 18:26:16.106329 22887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:38573 every 8 connection(s)
I20260807 18:26:16.107612 22710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:16.110244 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 22710
I20260807 18:26:16.110498 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:16.111129 30323 external_mini_cluster.cc:1428] Setting key dc61af115dc3893e9e523408cb67da7d
Aug 07 18:26:16 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127176, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:16.129931 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.110476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:54625 (local address 127.29.156.254:44997)
0807 18:26:16.111035 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.111043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.112237 (+ 1194us) server_negotiation.cc:407] Connection header received
0807 18:26:16.114160 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.114166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.114251 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.114381 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:16.115968 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.116753 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.118286 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.118705 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.118847 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.122833 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:16.122838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:16.122842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:16.122880 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:16.126350 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.127522 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.127584 (+ 62us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.127588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.127739 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.128231 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.128234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.128237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.128531 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:16.128695 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.129417 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.129635 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":268,"threads_started":1}
I20260807 18:26:16.130942 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.111142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44997 (local address 127.29.156.195:54625)
0807 18:26:16.112075 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.112124 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.113882 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.114616 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.114624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.115149 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:16.115793 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.115813 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.116911 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.116927 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.118138 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.118150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.118944 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.118952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.119178 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.119924 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:16.119949 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:16.122601 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:16.126544 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:16.126555 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:16.126578 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:16.127346 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:16.127873 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:16.127876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:16.127879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:16.128092 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:16.128684 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:16.128691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:16.129031 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.129270 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.129681 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":404,"threads_started":1}
I20260807 18:26:16.132401 22889 heartbeater.cc:347] Connected to a master server at 127.29.156.254:44997
I20260807 18:26:16.132781 22889 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:16.133600 22889 heartbeater.cc:511] Master 127.29.156.254:44997 requested a full tablet report, sending...
I20260807 18:26:16.135221 22223 ts_manager.cc:194] Registered new tserver with Master: f6b5480504e94858aa937cd0345f3cbd (127.29.156.195:38573)
I20260807 18:26:16.136214 22223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:54625
I20260807 18:26:16.140569 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:16.161201 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.146091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.29.156.254:44997)
0807 18:26:16.146292 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.146297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.146689 (+ 392us) server_negotiation.cc:407] Connection header received
0807 18:26:16.146839 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.146843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.146912 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.147101 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:16.148886 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.149749 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.151411 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.151948 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.152058 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.154708 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:16.154713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:16.154717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:16.154754 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:16.157860 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.158762 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.158766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.158769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.158906 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.159360 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.159364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.159366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.159869 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:16.160165 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.160598 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.160839 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:26:16.166281 22223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:16.173492 22223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:16.208366 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.186593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:39069 (local address 127.0.0.1:42402)
0807 18:26:16.187390 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.187438 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.187688 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.199810 (+ 12122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.199821 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.199868 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.200480 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.200496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.202658 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.202666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.205359 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.205380 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.207223 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.207230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.207424 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.207487 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.207670 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.207881 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":4,"thread_start_us":716,"threads_started":1}
I20260807 18:26:16.209163 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.187133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.29.156.193:39069)
0807 18:26:16.198709 (+ 11576us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.198721 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.198765 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:26:16.198927 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.198942 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.199329 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.200742 (+ 1413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.201106 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.202508 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.205557 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.206987 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.207209 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.207393 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.208630 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.208819 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11341,"thread_start_us":11132,"threads_started":1}
I20260807 18:26:16.209993 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.188909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.29.156.195:38573)
0807 18:26:16.197999 (+ 9090us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.198014 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.200877 (+ 2863us) server_negotiation.cc:407] Connection header received
0807 18:26:16.201034 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.201042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.201321 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.201531 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.202450 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.203671 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.206306 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.207656 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.207852 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.208032 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.209439 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.209620 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8811,"thread_start_us":8581,"threads_started":1}
I20260807 18:26:16.210018 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.187409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.29.156.194:46057)
0807 18:26:16.200756 (+ 13347us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.200765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.200798 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:26:16.200940 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.200960 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.201242 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.201459 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.202567 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.204388 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.206717 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.208253 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.208416 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.208562 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.209423 (+ 861us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.209600 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13146,"thread_start_us":14227,"threads_started":1}
I20260807 18:26:16.209304 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.188219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38573 (local address 127.0.0.1:40168)
0807 18:26:16.200561 (+ 12342us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.200579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.200721 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.201652 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.201658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.201683 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.202177 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.202196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.203862 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.203867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.206115 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.206131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.208196 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.208202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.208409 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.208453 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.208742 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.208867 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12203,"mutex_wait_us":27,"thread_start_us":11457,"threads_started":1}
I20260807 18:26:16.210628 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.186768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46057 (local address 127.0.0.1:43062)
0807 18:26:16.200004 (+ 13236us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.200038 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.200180 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.201854 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.201860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.201880 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.202390 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.202403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.204573 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.204581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.206507 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.206523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.208722 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.208739 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.208922 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.208967 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.209062 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.209186 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13070,"mutex_wait_us":50,"thread_start_us":13419,"threads_started":1}
I20260807 18:26:16.213971 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 821988d9a6854085a207c80167b30223 (DEFAULT_TABLE table=test-table [id=4ef895127a954cdf988e11d31db673dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:16.214130 22611 tablet_service.cc:1467] Processing CreateTablet for tablet 821988d9a6854085a207c80167b30223 (DEFAULT_TABLE table=test-table [id=4ef895127a954cdf988e11d31db673dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:16.214054 22415 tablet_service.cc:1467] Processing CreateTablet for tablet 821988d9a6854085a207c80167b30223 (DEFAULT_TABLE table=test-table [id=4ef895127a954cdf988e11d31db673dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:16.215477 22415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821988d9a6854085a207c80167b30223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:16.215488 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821988d9a6854085a207c80167b30223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:16.215495 22611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821988d9a6854085a207c80167b30223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:16.232296 22921 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Bootstrap starting.
I20260807 18:26:16.235809 22921 tablet_bootstrap.cc:654] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:16.236553 22922 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Bootstrap starting.
I20260807 18:26:16.237247 22921 log.cc:824] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:16.241074 22922 tablet_bootstrap.cc:654] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:16.242352 22922 log.cc:824] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:16.245698 22922 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: No bootstrap required, opened a new log
I20260807 18:26:16.245749 22921 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: No bootstrap required, opened a new log
I20260807 18:26:16.245983 22922 ts_tablet_manager.cc:1397] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:26:16.246029 22921 ts_tablet_manager.cc:1397] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260807 18:26:16.256922 22928 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Bootstrap starting.
I20260807 18:26:16.257308 22922 raft_consensus.cc:348] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.257752 22922 raft_consensus.cc:374] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:16.257901 22922 raft_consensus.cc:729] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7be0b4ef6d421db89d96c9dcac566c, State: Initialized, Role: FOLLOWER
I20260807 18:26:16.258718 22922 consensus_queue.cc:260] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.259825 22921 raft_consensus.cc:348] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.260327 22921 raft_consensus.cc:374] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:16.260459 22921 raft_consensus.cc:729] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b5480504e94858aa937cd0345f3cbd, State: Initialized, Role: FOLLOWER
I20260807 18:26:16.260922 22922 ts_tablet_manager.cc:1428] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Time spent starting tablet: real 0.015s user 0.003s sys 0.011s
I20260807 18:26:16.261252 22921 consensus_queue.cc:260] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.262135 22928 tablet_bootstrap.cc:654] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:16.263495 22928 log.cc:824] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:16.265046 22889 heartbeater.cc:503] Master 127.29.156.254:44997 was elected leader, sending a full tablet report...
I20260807 18:26:16.265686 22921 ts_tablet_manager.cc:1428] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Time spent starting tablet: real 0.020s user 0.016s sys 0.000s
I20260807 18:26:16.268302 22928 tablet_bootstrap.cc:492] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: No bootstrap required, opened a new log
I20260807 18:26:16.268669 22928 ts_tablet_manager.cc:1397] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.007s
I20260807 18:26:16.277148 22928 raft_consensus.cc:348] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.277549 22928 raft_consensus.cc:374] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:16.277657 22928 raft_consensus.cc:729] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4b175fedfb4f2bacf715574623d3f8, State: Initialized, Role: FOLLOWER
I20260807 18:26:16.278298 22928 consensus_queue.cc:260] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.279623 22703 heartbeater.cc:503] Master 127.29.156.254:44997 was elected leader, sending a full tablet report...
I20260807 18:26:16.280251 22928 ts_tablet_manager.cc:1428] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260807 18:26:16.338775 22933 raft_consensus.cc:482] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:16.339149 22933 raft_consensus.cc:504] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.341065 22933 leader_election.cc:288] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c7be0b4ef6d421db89d96c9dcac566c (127.29.156.193:39069), f6b5480504e94858aa937cd0345f3cbd (127.29.156.195:38573)
I20260807 18:26:16.346486 22930 raft_consensus.cc:482] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:16.346818 22930 raft_consensus.cc:504] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.349080 22930 leader_election.cc:288] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c7be0b4ef6d421db89d96c9dcac566c (127.29.156.193:39069), aa4b175fedfb4f2bacf715574623d3f8 (127.29.156.194:46057)
I20260807 18:26:16.349581 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.341729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:50017 (local address 127.29.156.193:39069)
0807 18:26:16.342043 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.342051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.342328 (+ 277us) server_negotiation.cc:407] Connection header received
0807 18:26:16.342713 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.342720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.342816 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.342877 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.343750 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.344533 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.346997 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.348231 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.348450 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.348536 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.348984 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.349166 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:26:16.349717 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.342673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:36901 (local address 127.29.156.195:38573)
0807 18:26:16.342932 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.342939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.342961 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:26:16.343226 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.343234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.343348 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.343458 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.344150 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.344898 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.347036 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.348353 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.348517 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.348599 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.349202 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.349362 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:26:16.350745 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.342175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38573 (local address 127.29.156.194:36901)
0807 18:26:16.342839 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.342859 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.342987 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.343651 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.343655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.343688 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.343987 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.343994 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.345070 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.345075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.346801 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.346819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.348730 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.348737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.349000 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.349034 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.349939 (+ 905us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.350126 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":31,"thread_start_us":350,"threads_started":1}
I20260807 18:26:16.350893 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.341366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:39069 (local address 127.29.156.194:50017)
0807 18:26:16.342165 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.342184 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.342494 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.343057 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.343060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.343083 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.343564 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.343579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.344692 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.344700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.346806 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.346830 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.348478 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.348485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.348729 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.348774 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.350017 (+ 1243us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.350150 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":58,"thread_start_us":322,"threads_started":1}
I20260807 18:26:16.352109 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "aa4b175fedfb4f2bacf715574623d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b5480504e94858aa937cd0345f3cbd" is_pre_election: true
I20260807 18:26:16.352604 22833 raft_consensus.cc:2454] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4b175fedfb4f2bacf715574623d3f8 in term 0.
I20260807 18:26:16.352795 22449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "aa4b175fedfb4f2bacf715574623d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" is_pre_election: true
I20260807 18:26:16.353539 22449 raft_consensus.cc:2454] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4b175fedfb4f2bacf715574623d3f8 in term 0.
I20260807 18:26:16.353747 22546 leader_election.cc:302] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa4b175fedfb4f2bacf715574623d3f8, f6b5480504e94858aa937cd0345f3cbd; no voters:
I20260807 18:26:16.354362 22933 raft_consensus.cc:2781] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:16.354552 22933 raft_consensus.cc:482] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:16.354681 22933 raft_consensus.cc:3037] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:16.358217 22933 raft_consensus.cc:504] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.358700 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.350386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:35007 (local address 127.29.156.194:46057)
0807 18:26:16.350588 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.350595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.351758 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 18:26:16.352090 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.352098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.352211 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.352292 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.353327 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.354429 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.356717 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.357618 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.357809 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.357906 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.358263 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.358396 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:26:16.358911 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.349564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46057 (local address 127.29.156.195:35007)
0807 18:26:16.351647 (+ 2083us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.351667 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.351899 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.352515 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.352519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.352546 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.353087 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.353102 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.354619 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.354625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.356552 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.356565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.357846 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.357853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.358044 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.358077 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.358230 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.358382 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1909,"mutex_wait_us":30,"thread_start_us":1641,"threads_started":1}
I20260807 18:26:16.358964 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.349233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:39069 (local address 127.29.156.195:45361)
0807 18:26:16.349595 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:16.349614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:16.349733 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:16.351756 (+ 2023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:16.351761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:16.351795 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:16.352057 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.352071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.353824 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.353830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.355960 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:16.355976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.357920 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.357930 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:16.358192 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.358238 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:26:16.358322 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:26:16.358483 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":1,"spinlock_wait_cycles":25472}
W20260807 18:26:16.359369 22890 tablet.cc:2369] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:16.360064 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.350171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:45361 (local address 127.29.156.193:39069)
0807 18:26:16.350759 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.350768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.350790 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:26:16.351460 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.351464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.351593 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.352222 (+ 629us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:16.352380 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.353623 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.356143 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.357661 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.359389 (+ 1728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.359483 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.359577 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.359757 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446}
I20260807 18:26:16.360358 22933 leader_election.cc:288] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c7be0b4ef6d421db89d96c9dcac566c (127.29.156.193:39069), f6b5480504e94858aa937cd0345f3cbd (127.29.156.195:38573)
I20260807 18:26:16.360581 22449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "aa4b175fedfb4f2bacf715574623d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7be0b4ef6d421db89d96c9dcac566c"
I20260807 18:26:16.360323 22641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "f6b5480504e94858aa937cd0345f3cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4b175fedfb4f2bacf715574623d3f8" is_pre_election: true
I20260807 18:26:16.360826 22449 raft_consensus.cc:3037] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:16.361192 22641 raft_consensus.cc:2379] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6b5480504e94858aa937cd0345f3cbd in current term 1: Already voted for candidate aa4b175fedfb4f2bacf715574623d3f8 in this term.
I20260807 18:26:16.361125 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "aa4b175fedfb4f2bacf715574623d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b5480504e94858aa937cd0345f3cbd"
I20260807 18:26:16.361374 22833 raft_consensus.cc:3037] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:16.362208 22447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821988d9a6854085a207c80167b30223" candidate_uuid: "f6b5480504e94858aa937cd0345f3cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" is_pre_election: true
I20260807 18:26:16.363229 22736 leader_election.cc:302] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6b5480504e94858aa937cd0345f3cbd; no voters: 3c7be0b4ef6d421db89d96c9dcac566c, aa4b175fedfb4f2bacf715574623d3f8
I20260807 18:26:16.364938 22449 raft_consensus.cc:2454] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4b175fedfb4f2bacf715574623d3f8 in term 1.
I20260807 18:26:16.365116 22833 raft_consensus.cc:2454] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4b175fedfb4f2bacf715574623d3f8 in term 1.
I20260807 18:26:16.365350 22930 raft_consensus.cc:2726] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 18:26:16.365417 22546 leader_election.cc:302] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7be0b4ef6d421db89d96c9dcac566c, aa4b175fedfb4f2bacf715574623d3f8; no voters:
I20260807 18:26:16.365728 22933 raft_consensus.cc:2781] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:16.366632 22933 raft_consensus.cc:686] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 LEADER]: Becoming Leader. State: Replica: aa4b175fedfb4f2bacf715574623d3f8, State: Running, Role: LEADER
I20260807 18:26:16.367208 22933 consensus_queue.cc:237] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } }
I20260807 18:26:16.372062 22221 catalog_manager.cc:5495] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa4b175fedfb4f2bacf715574623d3f8 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "aa4b175fedfb4f2bacf715574623d3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4b175fedfb4f2bacf715574623d3f8" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260807 18:26:16.419155 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.412711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.29.156.194:46057)
0807 18:26:16.412934 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.412943 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.412970 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:26:16.413277 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.413282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.413371 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.413552 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:16.414000 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.414856 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.416667 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.417123 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.417282 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.417906 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:16.418445 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:16.418661 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.418717 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.418811 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:26:16.421437 22514 tablet.cc:2369] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:16.428128 22449 raft_consensus.cc:1261] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 1 FOLLOWER]: Refusing update from remote peer aa4b175fedfb4f2bacf715574623d3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:16.428170 22833 raft_consensus.cc:1261] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 1 FOLLOWER]: Refusing update from remote peer aa4b175fedfb4f2bacf715574623d3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:16.429112 22937 consensus_queue.cc:1035] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c7be0b4ef6d421db89d96c9dcac566c" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 39069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:16.429564 22933 consensus_queue.cc:1035] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b5480504e94858aa937cd0345f3cbd" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:16.442003 22941 mvcc.cc:205] Tried to move back new op lower bound from 7315976914639335424 to 7315976914404741120. Current Snapshot: MvccSnapshot[applied={T|T < 7315976914639335424}]
I20260807 18:26:16.460512 22223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:16.460831 22223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:16.465102 22223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 991b8fd7aa8b4c1f8b44b7d0abb166fb: Sending DeleteTablet for 3 replicas of tablet 821988d9a6854085a207c80167b30223
I20260807 18:26:16.466487 22611 tablet_service.cc:1514] Processing DeleteTablet for tablet 821988d9a6854085a207c80167b30223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:16 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:43062
I20260807 18:26:16.466535 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 821988d9a6854085a207c80167b30223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:16 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:40168
I20260807 18:26:16.467494 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 22318
I20260807 18:26:16.467767 22415 tablet_service.cc:1514] Processing DeleteTablet for tablet 821988d9a6854085a207c80167b30223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:16 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:42402
I20260807 18:26:16.470585 22958 tablet_replica.cc:330] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: stopping tablet replica
I20260807 18:26:16.471287 22958 raft_consensus.cc:2229] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:16.472807 22958 raft_consensus.cc:2258] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:16.473685 22959 tablet_replica.cc:330] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: stopping tablet replica
I20260807 18:26:16.474385 22959 raft_consensus.cc:2229] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:16.477741 22958 ts_tablet_manager.cc:1905] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:16.478549 22959 raft_consensus.cc:2258] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:16.480428 22961 tablet_replica.cc:330] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: stopping tablet replica
I20260807 18:26:16.481033 22961 raft_consensus.cc:2229] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:16.481621 22961 raft_consensus.cc:2258] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:16.483870 22961 ts_tablet_manager.cc:1905] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:16.488246 22959 ts_tablet_manager.cc:1905] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:16.489468 22958 ts_tablet_manager.cc:1918] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:16.489766 22958 log.cc:1196] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/821988d9a6854085a207c80167b30223
I20260807 18:26:16.490458 22958 ts_tablet_manager.cc:1939] T 821988d9a6854085a207c80167b30223 P aa4b175fedfb4f2bacf715574623d3f8: Deleting consensus metadata
I20260807 18:26:16.491361 22961 ts_tablet_manager.cc:1918] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:16.491602 22961 log.cc:1196] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/821988d9a6854085a207c80167b30223
Aug 07 18:26:16 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:26:16.492208 22961 ts_tablet_manager.cc:1939] T 821988d9a6854085a207c80167b30223 P 3c7be0b4ef6d421db89d96c9dcac566c: Deleting consensus metadata
I20260807 18:26:16.492805 22199 catalog_manager.cc:4841] TS aa4b175fedfb4f2bacf715574623d3f8 (127.29.156.194:46057): tablet 821988d9a6854085a207c80167b30223 (table test-table [id=4ef895127a954cdf988e11d31db673dd]) successfully deleted
I20260807 18:26:16.493994 22197 catalog_manager.cc:4841] TS 3c7be0b4ef6d421db89d96c9dcac566c (127.29.156.193:39069): tablet 821988d9a6854085a207c80167b30223 (table test-table [id=4ef895127a954cdf988e11d31db673dd]) successfully deleted
I20260807 18:26:16.499471 22959 ts_tablet_manager.cc:1918] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:16.499711 22959 log.cc:1196] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/821988d9a6854085a207c80167b30223
I20260807 18:26:16.500308 22959 ts_tablet_manager.cc:1939] T 821988d9a6854085a207c80167b30223 P f6b5480504e94858aa937cd0345f3cbd: Deleting consensus metadata
I20260807 18:26:16.502341 22197 catalog_manager.cc:4841] TS f6b5480504e94858aa937cd0345f3cbd (127.29.156.195:38573): tablet 821988d9a6854085a207c80167b30223 (table test-table [id=4ef895127a954cdf988e11d31db673dd]) successfully deleted
I20260807 18:26:16.503769 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.469814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.29.156.193:39069)
0807 18:26:16.470001 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.470008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.479594 (+ 9586us) server_negotiation.cc:407] Connection header received
0807 18:26:16.480235 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.480244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.480342 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.480429 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:16.483096 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.484255 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.486322 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.487005 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.487110 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.496675 (+ 9565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:16.496688 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:16.496703 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:16.496740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:16.500338 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.501254 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.501264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.501267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.501428 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.501876 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.501881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.501885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.502390 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:16.502730 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.503082 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.503371 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:26:16.504742 22507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:16.616654 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 22318
I20260807 18:26:16.652192 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 22521
Aug 07 18:26:16 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:26:16.672034 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.653073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.29.156.194:46057)
0807 18:26:16.653264 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.653268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.653286 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:26:16.653504 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.653509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.653633 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.653729 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:16.655616 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.656489 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.658491 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.659028 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.659223 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.664312 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:16.664332 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:16.664349 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:16.664400 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:16.668481 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.669417 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.669427 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.669430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.669605 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.670078 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.670087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.670091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.670559 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:16.670953 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.671314 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.671648 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:26:16.672964 22698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:16.792161 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 22521
I20260807 18:26:16.818115 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 22710
Aug 07 18:26:16 dist-test-slave-sb2z krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:26:16.835708 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:16.819020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.29.156.195:38573)
0807 18:26:16.819207 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:16.819212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:16.819231 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:26:16.819432 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:16.819436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:16.819513 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:16.819665 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:16.821191 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.821937 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.823381 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:16.823966 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:16.824077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:16.828975 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:16.828994 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:16.829011 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:16.829101 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:16.832408 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.833216 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.833223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.833226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.833386 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:16.833871 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:16.833875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:16.833877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:16.834366 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:16.834601 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:16.835007 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 18:26:16.835195 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:26:16.836373 22881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:16.933430 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 22710
I20260807 18:26:16.959947 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 22176
I20260807 18:26:16.961282 22270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:17.075835 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 22176
2026-08-07T18:26:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5042 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:26:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:17Z Disabled control of system clock
I20260807 18:26:17.182165 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46279
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:17.544808 22973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:17.545104 22973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:17.545253 22973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:17.552667 22973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:17.552788 22973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:17.552879 22973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:17.552984 22973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:17.553066 22973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:17.562564 22973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:17.563977 22973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:17.565934 22973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:17.578486 22980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:17.578357 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:17.578440 22981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:17.579293 22973 server_base.cc:1034] running on GCE node
I20260807 18:26:17.580176 22973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:17.581617 22973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:17.582834 22973 hybrid_clock.cc:648] HybridClock initialized: now 1786127177582786 us; error 49 us; skew 500 ppm
I20260807 18:26:17.583238 22973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:17.586021 22973 webserver.cc:460] Webserver started at http://127.29.156.254:40183/ using document root <none> and password file <none>
I20260807 18:26:17.586757 22973 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:17.586851 22973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:17.587240 22973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:17.590348 22973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "a9a9ce89b17549ee81937d41dab2546a"
format_stamp: "Formatted at 2026-08-07 18:26:17 on dist-test-slave-sb2z"
server_key: "3d82e5f2560b1bb475f2331e9ee687dd"
server_key_iv: "cab65e738aad9a9604ae56a13730e137"
server_key_version: "encryptionkey@0"
I20260807 18:26:17.591444 22973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "a9a9ce89b17549ee81937d41dab2546a"
format_stamp: "Formatted at 2026-08-07 18:26:17 on dist-test-slave-sb2z"
server_key: "3d82e5f2560b1bb475f2331e9ee687dd"
server_key_iv: "cab65e738aad9a9604ae56a13730e137"
server_key_version: "encryptionkey@0"
I20260807 18:26:17.597385 22973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 18:26:17.601047 22991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:17.602483 22973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:17.602694 22973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "a9a9ce89b17549ee81937d41dab2546a"
format_stamp: "Formatted at 2026-08-07 18:26:17 on dist-test-slave-sb2z"
server_key: "3d82e5f2560b1bb475f2331e9ee687dd"
server_key_iv: "cab65e738aad9a9604ae56a13730e137"
server_key_version: "encryptionkey@0"
I20260807 18:26:17.602936 22973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:17.622979 22973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:17.623962 22973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:17.624302 22973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:17.644580 22973 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:46279
I20260807 18:26:17.644603 23061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:46279 every 8 connection(s)
I20260807 18:26:17.645996 22973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:17.646580 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 22973
I20260807 18:26:17.646852 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:17.647578 30323 external_mini_cluster.cc:1428] Setting key 17a8cfd87c21319e5fd81934b4ccadf7
I20260807 18:26:17.651206 23062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:17.661441 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:17.649749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.29.156.254:46279)
0807 18:26:17.650802 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:17.650813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:17.650854 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:26:17.652652 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:17.652674 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:17.653094 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:17.653503 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:17.654084 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:17.655415 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:17.656886 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:17.657542 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:17.657753 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:17.658996 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:17.659004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:17.659017 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:17.659648 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:17.659725 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:17.659746 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:17.659901 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:17.660127 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:26:17.660425 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":401,"threads_started":1}
I20260807 18:26:17.663372 23062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: Bootstrap starting.
I20260807 18:26:17.666894 23062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:17.668313 23062 log.cc:824] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:17.671288 23062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: No bootstrap required, opened a new log
I20260807 18:26:17.679661 23062 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } }
I20260807 18:26:17.680094 23062 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:17.680219 23062 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a9ce89b17549ee81937d41dab2546a, State: Initialized, Role: FOLLOWER
I20260807 18:26:17.681027 23062 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } }
I20260807 18:26:17.681367 23062 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:17.681574 23062 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:17.681788 23062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:17.684337 23062 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } }
I20260807 18:26:17.684901 23062 leader_election.cc:302] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9a9ce89b17549ee81937d41dab2546a; no voters:
I20260807 18:26:17.685609 23062 leader_election.cc:288] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:17.685776 23070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:17.686729 23070 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [term 1 LEADER]: Becoming Leader. State: Replica: a9a9ce89b17549ee81937d41dab2546a, State: Running, Role: LEADER
I20260807 18:26:17.687305 23070 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } }
I20260807 18:26:17.688215 23062 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:17.690122 23072 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a9ce89b17549ee81937d41dab2546a. Latest consensus state: current_term: 1 leader_uuid: "a9a9ce89b17549ee81937d41dab2546a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } } }
I20260807 18:26:17.690529 23072 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:17.691042 23071 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9a9ce89b17549ee81937d41dab2546a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a9ce89b17549ee81937d41dab2546a" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46279 } } }
I20260807 18:26:17.691596 23071 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:17.696091 23078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:17.701956 23078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:17.712792 23078 catalog_manager.cc:1347] Generated new cluster ID: e0ffda095eca407b97be88df655370fa
I20260807 18:26:17.712949 23078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:17.723738 23078 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:17.725128 23078 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:17.733347 23078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: Generated new TSK 0
I20260807 18:26:17.734267 23078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:17.752159 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--builtin_ntp_servers=127.29.156.212:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:18.140372 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:18.140686 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:18.140776 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:18.148331 23102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:18.148454 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:18.148576 23102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:18.158053 23102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:18.159343 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:18.161216 23102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:18.176026 23107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:18.176079 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:18.176105 23109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:18.176785 23102 server_base.cc:1034] running on GCE node
I20260807 18:26:18.177436 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:18.178522 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:18.179767 23102 hybrid_clock.cc:648] HybridClock initialized: now 1786127178179727 us; error 55 us; skew 500 ppm
I20260807 18:26:18.180123 23102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:18.182677 23102 webserver.cc:460] Webserver started at http://127.29.156.193:42801/ using document root <none> and password file <none>
I20260807 18:26:18.183426 23102 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:18.183624 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:18.183970 23102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:18.187093 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "c64ecb6c745441ad85b3e746e2548ff8"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "067ae80b1966600d006d7a5d38dd0a23"
server_key_iv: "e3e3dce0c14a8589350bc212da7c2a1c"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.188031 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "c64ecb6c745441ad85b3e746e2548ff8"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "067ae80b1966600d006d7a5d38dd0a23"
server_key_iv: "e3e3dce0c14a8589350bc212da7c2a1c"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.193500 23102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 18:26:18.197484 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.198967 23102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:26:18.199159 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "c64ecb6c745441ad85b3e746e2548ff8"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "067ae80b1966600d006d7a5d38dd0a23"
server_key_iv: "e3e3dce0c14a8589350bc212da7c2a1c"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.199393 23102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:18.234865 23102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:18.235826 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:18.236176 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:18.237277 23102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:18.238878 23102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:18.239015 23102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.239161 23102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:18.239254 23102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.276324 23102 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:35033
I20260807 18:26:18.276427 23282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:35033 every 8 connection(s)
I20260807 18:26:18.277705 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:18.284373 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 23102
I20260807 18:26:18.284641 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:18.285300 30323 external_mini_cluster.cc:1428] Setting key 2c50c221334c4a272a47507712f72009
I20260807 18:26:18.288718 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--builtin_ntp_servers=127.29.156.212:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:18.293893 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:18.280718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:59235 (local address 127.29.156.254:46279)
0807 18:26:18.281240 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:18.281245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:18.282623 (+ 1378us) server_negotiation.cc:407] Connection header received
0807 18:26:18.285021 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:18.285029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:18.285131 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:18.285257 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:18.286710 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.287801 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.290050 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.290836 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.291002 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:18.292797 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:18.292806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:18.292813 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:18.292841 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:18.292898 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:18.292917 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:18.293018 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:18.293325 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 18:26:18.293442 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":271,"threads_started":1}
I20260807 18:26:18.295030 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:18.281365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46279 (local address 127.29.156.193:59235)
0807 18:26:18.282387 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:18.282470 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:18.284504 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:18.285601 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:18.285610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:18.285710 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:18.286496 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:18.286517 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.288007 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.288011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:18.289861 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:18.289891 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.291165 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.291172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:18.291427 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:18.292407 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:18.292410 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:18.292457 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:18.292553 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:18.293063 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:18.293074 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:18.293081 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:26:18.293276 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 18:26:18.293630 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":317,"threads_started":1}
I20260807 18:26:18.296972 23283 heartbeater.cc:347] Connected to a master server at 127.29.156.254:46279
I20260807 18:26:18.297334 23283 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:18.298346 23283 heartbeater.cc:511] Master 127.29.156.254:46279 requested a full tablet report, sending...
I20260807 18:26:18.300809 23014 ts_manager.cc:194] Registered new tserver with Master: c64ecb6c745441ad85b3e746e2548ff8 (127.29.156.193:35033)
I20260807 18:26:18.302356 23014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.193:59235
W20260807 18:26:18.645671 23292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:18.645951 23292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:18.646095 23292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:18.653357 23292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:18.653460 23292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:18.653652 23292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:18.662842 23292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:18.664207 23292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:18.666049 23292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:18.680088 23299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:18.680307 23298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:18.680811 23302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:18.681159 23292 server_base.cc:1034] running on GCE node
I20260807 18:26:18.681731 23292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:18.682701 23292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:18.683910 23292 hybrid_clock.cc:648] HybridClock initialized: now 1786127178683839 us; error 60 us; skew 500 ppm
I20260807 18:26:18.684301 23292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:18.686796 23292 webserver.cc:460] Webserver started at http://127.29.156.194:41019/ using document root <none> and password file <none>
I20260807 18:26:18.687625 23292 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:18.687757 23292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:18.688113 23292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:18.691094 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "926aaf8b915642b6a462c2262d227d3c"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "844bdcd3b74a459bde887594a11df3a1"
server_key_iv: "34708bd5cc0fe010d30ce778dd48787f"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.691970 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "926aaf8b915642b6a462c2262d227d3c"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "844bdcd3b74a459bde887594a11df3a1"
server_key_iv: "34708bd5cc0fe010d30ce778dd48787f"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.697556 23292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 18:26:18.701833 23312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.703298 23292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:26:18.703483 23292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "926aaf8b915642b6a462c2262d227d3c"
format_stamp: "Formatted at 2026-08-07 18:26:18 on dist-test-slave-sb2z"
server_key: "844bdcd3b74a459bde887594a11df3a1"
server_key_iv: "34708bd5cc0fe010d30ce778dd48787f"
server_key_version: "encryptionkey@0"
I20260807 18:26:18.703784 23292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:18.746760 23292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:18.747777 23292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:18.748145 23292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:18.749254 23292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:18.750901 23292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:18.751045 23292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.751204 23292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:18.751304 23292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:18.788998 23292 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:42309
I20260807 18:26:18.789049 23476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:42309 every 8 connection(s)
I20260807 18:26:18.790558 23292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:18.796765 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 23292
I20260807 18:26:18.797015 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:18.797648 30323 external_mini_cluster.cc:1428] Setting key ae61f6f99d606fb1f4a25fbe8b37d98b
I20260807 18:26:18.801069 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--builtin_ntp_servers=127.29.156.212:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:18.806330 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:18.793530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:42193 (local address 127.29.156.254:46279)
0807 18:26:18.793764 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:18.793768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:18.795570 (+ 1802us) server_negotiation.cc:407] Connection header received
0807 18:26:18.797861 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:18.797870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:18.797996 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:18.798114 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:18.799449 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.800543 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.802444 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.803210 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.803368 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:18.805249 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:18.805258 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:18.805265 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:18.805293 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:18.805337 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:18.805356 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:18.805468 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:18.805781 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 18:26:18.805913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 18:26:18.807503 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:18.794074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46279 (local address 127.29.156.194:42193)
0807 18:26:18.795286 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:18.795326 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:18.797507 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:18.798381 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:18.798389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:18.798522 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:18.799266 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:18.799285 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.800722 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.800727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:18.802270 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:18.802285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:18.803509 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:18.803515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:18.803891 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:18.804863 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:18.804866 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:18.804915 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:18.805007 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:18.805487 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:18.805499 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:18.805506 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:26:18.805735 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 18:26:18.806127 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":446,"threads_started":1}
I20260807 18:26:18.809399 23477 heartbeater.cc:347] Connected to a master server at 127.29.156.254:46279
I20260807 18:26:18.809759 23477 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:18.810845 23477 heartbeater.cc:511] Master 127.29.156.254:46279 requested a full tablet report, sending...
I20260807 18:26:18.812623 23014 ts_manager.cc:194] Registered new tserver with Master: 926aaf8b915642b6a462c2262d227d3c (127.29.156.194:42309)
I20260807 18:26:18.813581 23014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.194:42193
W20260807 18:26:19.165037 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:19.165407 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:19.165591 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:19.173354 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:19.173501 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:19.173683 23483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:19.183203 23483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:19.184656 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:19.186592 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:19.200730 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:19.200750 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:19.200784 23497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:19.201848 23483 server_base.cc:1034] running on GCE node
I20260807 18:26:19.202713 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:19.204021 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:19.205312 23483 hybrid_clock.cc:648] HybridClock initialized: now 1786127179205272 us; error 58 us; skew 500 ppm
I20260807 18:26:19.205690 23483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:19.208705 23483 webserver.cc:460] Webserver started at http://127.29.156.195:38733/ using document root <none> and password file <none>
I20260807 18:26:19.209506 23483 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:19.209651 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:19.210002 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:19.213362 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "563070729b3848079b5e183a6456e04d"
format_stamp: "Formatted at 2026-08-07 18:26:19 on dist-test-slave-sb2z"
server_key: "9a05e4d223762961d2d5005fbe29d3ec"
server_key_iv: "e4683257b6ebccc19525d0db3bbe86cb"
server_key_version: "encryptionkey@0"
I20260807 18:26:19.214221 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "563070729b3848079b5e183a6456e04d"
format_stamp: "Formatted at 2026-08-07 18:26:19 on dist-test-slave-sb2z"
server_key: "9a05e4d223762961d2d5005fbe29d3ec"
server_key_iv: "e4683257b6ebccc19525d0db3bbe86cb"
server_key_version: "encryptionkey@0"
I20260807 18:26:19.219899 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:26:19.223723 23508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:19.225345 23483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:19.225560 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "563070729b3848079b5e183a6456e04d"
format_stamp: "Formatted at 2026-08-07 18:26:19 on dist-test-slave-sb2z"
server_key: "9a05e4d223762961d2d5005fbe29d3ec"
server_key_iv: "e4683257b6ebccc19525d0db3bbe86cb"
server_key_version: "encryptionkey@0"
I20260807 18:26:19.225805 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:19.245091 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:19.246006 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:19.246344 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:19.247452 23483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:19.249063 23483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:19.249202 23483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:19.249341 23483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:19.249449 23483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:19.289740 23483 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33989
I20260807 18:26:19.289848 23662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33989 every 8 connection(s)
I20260807 18:26:19.291287 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:19.301637 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 23483
I20260807 18:26:19.301844 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:19.302444 30323 external_mini_cluster.cc:1428] Setting key b02fcef8095c034bf8ff2a759403f9c6
I20260807 18:26:19.305688 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.294339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33649 (local address 127.29.156.254:46279)
0807 18:26:19.294572 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.294577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.296099 (+ 1522us) server_negotiation.cc:407] Connection header received
0807 18:26:19.298106 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.298138 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.298230 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.298323 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:19.299432 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.300246 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.302042 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.302499 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.302666 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.304721 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:19.304731 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:19.304736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:19.304765 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:19.304807 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:19.304830 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:19.304947 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.305216 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.305320 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:26:19.305996 23283 heartbeater.cc:503] Master 127.29.156.254:46279 was elected leader, sending a full tablet report...
I20260807 18:26:19.307369 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.294820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:46279 (local address 127.29.156.195:33649)
0807 18:26:19.295914 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.295953 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.297764 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.298551 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.298560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.298660 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:19.299264 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.299283 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.300419 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.300423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.301808 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.301829 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.302773 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.302783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.303123 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.304221 (+ 1098us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:19.304226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:19.304290 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:19.304423 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:19.304980 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:19.304990 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:19.304998 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.305166 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.305515 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":313,"threads_started":1}
I20260807 18:26:19.309082 23665 heartbeater.cc:347] Connected to a master server at 127.29.156.254:46279
I20260807 18:26:19.309438 23665 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:19.310336 23665 heartbeater.cc:511] Master 127.29.156.254:46279 requested a full tablet report, sending...
I20260807 18:26:19.311836 23014 ts_manager.cc:194] Registered new tserver with Master: 563070729b3848079b5e183a6456e04d (127.29.156.195:33989)
I20260807 18:26:19.312664 23014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.195:33649
I20260807 18:26:19.319339 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:19.332401 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.325649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.29.156.254:46279)
0807 18:26:19.325997 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.326005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.326485 (+ 480us) server_negotiation.cc:407] Connection header received
0807 18:26:19.326656 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.326660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.326723 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.326806 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:19.327354 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.328146 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.329683 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.330177 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.330325 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.331655 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:19.331660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:19.331663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:19.331681 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:19.331713 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:19.331724 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:19.331822 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.331970 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.332067 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:26:19.336679 23014 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:19.344349 23014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:19.387398 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.358038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35033 (local address 127.0.0.1:51172)
0807 18:26:19.376509 (+ 18471us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.376530 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.376666 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.378283 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.378291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.378323 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:19.378819 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.378836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.380998 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.381003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.383911 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.383930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.386116 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.386124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.386482 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.386547 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.386701 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.386934 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18304,"mutex_wait_us":59,"thread_start_us":13784,"threads_started":1}
I20260807 18:26:19.387431 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.358611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.29.156.193:35033)
0807 18:26:19.377252 (+ 18641us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.377262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.377297 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:26:19.377403 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.377411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.377760 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.377982 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:19.379066 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.380793 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.384211 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.385886 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.386090 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.386468 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.386784 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.387007 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18431,"thread_start_us":18265,"threads_started":1}
I20260807 18:26:19.388868 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.357790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33989 (local address 127.0.0.1:55894)
0807 18:26:19.372283 (+ 14493us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.372328 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.372546 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.378926 (+ 6380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.378934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.378978 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:19.379389 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.379403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.382314 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.382321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.385125 (+ 2804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.385139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.387961 (+ 2822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.387972 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.388113 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.388139 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.388337 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.388482 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14204,"mutex_wait_us":32,"thread_start_us":22695,"threads_started":1}
I20260807 18:26:19.388970 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.358097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.29.156.195:33989)
0807 18:26:19.378015 (+ 19918us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.378035 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.378072 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:26:19.378166 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.378174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.378403 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.378616 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:19.379788 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.382146 (+ 2358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.385392 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.386894 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.387115 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.387374 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.388408 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.388601 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19708,"thread_start_us":19565,"threads_started":1}
I20260807 18:26:19.390051 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.359984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:42309 (local address 127.0.0.1:43834)
0807 18:26:19.380827 (+ 20843us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.380847 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.381131 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.381876 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.381880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.381907 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:19.382388 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.382401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.383794 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.383801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.386324 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.386341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.389025 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.389032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.389398 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.389433 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.389536 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.389680 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20701,"thread_start_us":16164,"threads_started":1}
I20260807 18:26:19.391875 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.360711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.29.156.194:42309)
0807 18:26:19.380088 (+ 19377us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.380102 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.380993 (+ 891us) server_negotiation.cc:407] Connection header received
0807 18:26:19.381410 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.381418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.381659 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.381872 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:19.382589 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.383640 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.386606 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.388462 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.388640 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.388861 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.390292 (+ 1431us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.390463 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19128,"thread_start_us":19400,"threads_started":1}
I20260807 18:26:19.392274 23187 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a (DEFAULT_TABLE table=test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:19.394032 23187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2928f160944de79f3b8d89cd4c8a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:19.393440 23575 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a (DEFAULT_TABLE table=test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:19.395120 23575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2928f160944de79f3b8d89cd4c8a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:19.395409 23385 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a (DEFAULT_TABLE table=test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:19.397220 23385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2928f160944de79f3b8d89cd4c8a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:19.410161 23694 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Bootstrap starting.
I20260807 18:26:19.413440 23695 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Bootstrap starting.
I20260807 18:26:19.415555 23694 tablet_bootstrap.cc:654] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:19.416371 23697 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Bootstrap starting.
I20260807 18:26:19.417124 23694 log.cc:824] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:19.417496 23695 tablet_bootstrap.cc:654] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:19.418692 23695 log.cc:824] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:19.420048 23697 tablet_bootstrap.cc:654] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:19.420696 23694 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: No bootstrap required, opened a new log
I20260807 18:26:19.421016 23694 ts_tablet_manager.cc:1397] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 18:26:19.421355 23697 log.cc:824] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:19.422314 23695 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: No bootstrap required, opened a new log
I20260807 18:26:19.422670 23695 ts_tablet_manager.cc:1397] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 18:26:19.426222 23697 tablet_bootstrap.cc:492] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: No bootstrap required, opened a new log
I20260807 18:26:19.426523 23697 ts_tablet_manager.cc:1397] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 18:26:19.431020 23695 raft_consensus.cc:348] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.431396 23695 raft_consensus.cc:374] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:19.431490 23695 raft_consensus.cc:729] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64ecb6c745441ad85b3e746e2548ff8, State: Initialized, Role: FOLLOWER
I20260807 18:26:19.432333 23695 consensus_queue.cc:260] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.434053 23695 ts_tablet_manager.cc:1428] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
I20260807 18:26:19.433280 23694 raft_consensus.cc:348] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.434882 23694 raft_consensus.cc:374] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:19.435029 23694 raft_consensus.cc:729] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926aaf8b915642b6a462c2262d227d3c, State: Initialized, Role: FOLLOWER
I20260807 18:26:19.435727 23694 consensus_queue.cc:260] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.438108 23477 heartbeater.cc:503] Master 127.29.156.254:46279 was elected leader, sending a full tablet report...
I20260807 18:26:19.437940 23697 raft_consensus.cc:348] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.438329 23694 ts_tablet_manager.cc:1428] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 18:26:19.438412 23697 raft_consensus.cc:374] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:19.438526 23697 raft_consensus.cc:729] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563070729b3848079b5e183a6456e04d, State: Initialized, Role: FOLLOWER
I20260807 18:26:19.439145 23697 consensus_queue.cc:260] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.440419 23665 heartbeater.cc:503] Master 127.29.156.254:46279 was elected leader, sending a full tablet report...
I20260807 18:26:19.442425 23697 ts_tablet_manager.cc:1428] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 18:26:19.515384 23706 raft_consensus.cc:482] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:19.515734 23706 raft_consensus.cc:504] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.517488 23706 leader_election.cc:288] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 563070729b3848079b5e183a6456e04d (127.29.156.195:33989), c64ecb6c745441ad85b3e746e2548ff8 (127.29.156.193:35033)
I20260807 18:26:19.527307 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.518096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:47507 (local address 127.29.156.193:35033)
0807 18:26:19.518579 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.518589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.518913 (+ 324us) server_negotiation.cc:407] Connection header received
0807 18:26:19.519174 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.519182 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.519282 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.519413 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:19.520386 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.521547 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.524774 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.526029 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.526229 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.526289 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.526905 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.527049 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325}
I20260807 18:26:19.527354 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.518018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:50819 (local address 127.29.156.195:33989)
0807 18:26:19.518381 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.518391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.519138 (+ 747us) server_negotiation.cc:407] Connection header received
0807 18:26:19.519296 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.519301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.519385 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.519509 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:19.520386 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.521531 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.524774 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.526029 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.526216 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.526288 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.526976 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.527115 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 18:26:19.527449 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.517930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33989 (local address 127.29.156.194:50819)
0807 18:26:19.519025 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.519043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.519166 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.519499 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.519502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.519545 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:19.520201 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.520217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.521703 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.521707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.524517 (+ 2810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.524534 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.526326 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.526332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.526632 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.526725 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.526911 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.527048 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":260,"thread_start_us":700,"threads_started":1}
I20260807 18:26:19.528074 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.517786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:35033 (local address 127.29.156.194:47507)
0807 18:26:19.518712 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:19.518793 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:19.518976 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:19.519455 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:19.519462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:19.519492 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:19.520204 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.520217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.521701 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.521707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.524521 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:19.524534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.526320 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.526328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:19.526626 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.526726 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 18:26:19.527519 (+ 793us) client_negotiation.cc:240] Negotiation successful
0807 18:26:19.527699 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":330,"thread_start_us":502,"threads_started":1}
I20260807 18:26:19.528899 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2928f160944de79f3b8d89cd4c8a1a" candidate_uuid: "926aaf8b915642b6a462c2262d227d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563070729b3848079b5e183a6456e04d" is_pre_election: true
I20260807 18:26:19.529055 23215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2928f160944de79f3b8d89cd4c8a1a" candidate_uuid: "926aaf8b915642b6a462c2262d227d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64ecb6c745441ad85b3e746e2548ff8" is_pre_election: true
I20260807 18:26:19.529495 23602 raft_consensus.cc:2454] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926aaf8b915642b6a462c2262d227d3c in term 0.
I20260807 18:26:19.529678 23215 raft_consensus.cc:2454] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926aaf8b915642b6a462c2262d227d3c in term 0.
I20260807 18:26:19.530216 23318 leader_election.cc:302] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563070729b3848079b5e183a6456e04d, 926aaf8b915642b6a462c2262d227d3c; no voters:
W20260807 18:26:19.530471 23286 tablet.cc:2369] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:19.530730 23706 raft_consensus.cc:2781] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:19.530897 23706 raft_consensus.cc:482] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:19.531008 23706 raft_consensus.cc:3037] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:19.533413 23706 raft_consensus.cc:504] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
I20260807 18:26:19.533954 23706 leader_election.cc:288] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [CANDIDATE]: Term 1 election: Requested vote from peers 563070729b3848079b5e183a6456e04d (127.29.156.195:33989), c64ecb6c745441ad85b3e746e2548ff8 (127.29.156.193:35033)
I20260807 18:26:19.534497 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2928f160944de79f3b8d89cd4c8a1a" candidate_uuid: "926aaf8b915642b6a462c2262d227d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563070729b3848079b5e183a6456e04d"
I20260807 18:26:19.534495 23215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2928f160944de79f3b8d89cd4c8a1a" candidate_uuid: "926aaf8b915642b6a462c2262d227d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64ecb6c745441ad85b3e746e2548ff8"
I20260807 18:26:19.534720 23215 raft_consensus.cc:3037] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:19.534719 23602 raft_consensus.cc:3037] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:19.538128 23602 raft_consensus.cc:2454] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926aaf8b915642b6a462c2262d227d3c in term 1.
I20260807 18:26:19.538120 23215 raft_consensus.cc:2454] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926aaf8b915642b6a462c2262d227d3c in term 1.
I20260807 18:26:19.538640 23314 leader_election.cc:302] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 563070729b3848079b5e183a6456e04d, 926aaf8b915642b6a462c2262d227d3c, c64ecb6c745441ad85b3e746e2548ff8; no voters:
I20260807 18:26:19.539007 23706 raft_consensus.cc:2781] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:19.539592 23706 raft_consensus.cc:686] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 1 LEADER]: Becoming Leader. State: Replica: 926aaf8b915642b6a462c2262d227d3c, State: Running, Role: LEADER
I20260807 18:26:19.540189 23706 consensus_queue.cc:237] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } }
W20260807 18:26:19.543020 23479 tablet.cc:2369] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:19.543361 23666 tablet.cc:2369] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:19.545259 23014 catalog_manager.cc:5495] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 926aaf8b915642b6a462c2262d227d3c (127.29.156.194). New cstate: current_term: 1 leader_uuid: "926aaf8b915642b6a462c2262d227d3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926aaf8b915642b6a462c2262d227d3c" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 42309 } health_report { overall_health: HEALTHY } } }
I20260807 18:26:19.577739 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.571859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.29.156.194:42309)
0807 18:26:19.572078 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.572085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.572242 (+ 157us) server_negotiation.cc:407] Connection header received
0807 18:26:19.572537 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.572541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.572624 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.572790 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:19.573186 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.573919 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.575578 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.576056 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.576217 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.576703 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:19.577228 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:19.577369 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.577449 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.577520 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:26:19.586916 23602 raft_consensus.cc:1261] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 1 FOLLOWER]: Refusing update from remote peer 926aaf8b915642b6a462c2262d227d3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:19.586906 23215 raft_consensus.cc:1261] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 1 FOLLOWER]: Refusing update from remote peer 926aaf8b915642b6a462c2262d227d3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:19.588227 23712 consensus_queue.cc:1035] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "563070729b3848079b5e183a6456e04d" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:19.588652 23706 consensus_queue.cc:1035] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64ecb6c745441ad85b3e746e2548ff8" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 35033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:19.601889 23719 mvcc.cc:205] Tried to move back new op lower bound from 7315976927575351296 to 7315976927400906752. Current Snapshot: MvccSnapshot[applied={T|T < 7315976927575351296}]
I20260807 18:26:19.617764 23014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:19.618072 23014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:19.622033 23014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9a9ce89b17549ee81937d41dab2546a: Sending DeleteTablet for 3 replicas of tablet 1c2928f160944de79f3b8d89cd4c8a1a
I20260807 18:26:19.623332 23187 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:19 UTC) from {username='slave'} at 127.0.0.1:51172
I20260807 18:26:19.623332 23575 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:19 UTC) from {username='slave'} at 127.0.0.1:55894
I20260807 18:26:19.624351 23735 tablet_replica.cc:330] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: stopping tablet replica
I20260807 18:26:19.624502 23385 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2928f160944de79f3b8d89cd4c8a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:19 UTC) from {username='slave'} at 127.0.0.1:43834
I20260807 18:26:19.625012 23735 raft_consensus.cc:2229] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:19.625638 23735 raft_consensus.cc:2258] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:19.626538 23736 tablet_replica.cc:330] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: stopping tablet replica
I20260807 18:26:19.626688 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 23102
I20260807 18:26:19.626787 23737 tablet_replica.cc:330] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: stopping tablet replica
I20260807 18:26:19.627240 23737 raft_consensus.cc:2229] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:19.628168 23737 raft_consensus.cc:2258] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:19.628727 23736 raft_consensus.cc:2229] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:19.629491 23736 raft_consensus.cc:2258] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:19.630786 23735 ts_tablet_manager.cc:1905] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:19.632934 23737 ts_tablet_manager.cc:1905] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:19.636650 23736 ts_tablet_manager.cc:1905] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:19.638348 23735 ts_tablet_manager.cc:1918] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:19.638587 23735 log.cc:1196] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/1c2928f160944de79f3b8d89cd4c8a1a
I20260807 18:26:19.639206 23735 ts_tablet_manager.cc:1939] T 1c2928f160944de79f3b8d89cd4c8a1a P 563070729b3848079b5e183a6456e04d: Deleting consensus metadata
I20260807 18:26:19.641327 22995 catalog_manager.cc:4841] TS 563070729b3848079b5e183a6456e04d (127.29.156.195:33989): tablet 1c2928f160944de79f3b8d89cd4c8a1a (table test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]) successfully deleted
I20260807 18:26:19.643971 23736 ts_tablet_manager.cc:1918] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:19.644147 23736 log.cc:1196] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/1c2928f160944de79f3b8d89cd4c8a1a
I20260807 18:26:19.644685 23736 ts_tablet_manager.cc:1939] T 1c2928f160944de79f3b8d89cd4c8a1a P c64ecb6c745441ad85b3e746e2548ff8: Deleting consensus metadata
I20260807 18:26:19.646550 22992 catalog_manager.cc:4841] TS c64ecb6c745441ad85b3e746e2548ff8 (127.29.156.193:35033): tablet 1c2928f160944de79f3b8d89cd4c8a1a (table test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]) successfully deleted
I20260807 18:26:19.648228 23737 ts_tablet_manager.cc:1918] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:19.648432 23737 log.cc:1196] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/1c2928f160944de79f3b8d89cd4c8a1a
I20260807 18:26:19.649001 23737 ts_tablet_manager.cc:1939] T 1c2928f160944de79f3b8d89cd4c8a1a P 926aaf8b915642b6a462c2262d227d3c: Deleting consensus metadata
I20260807 18:26:19.649729 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.628457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.29.156.193:35033)
0807 18:26:19.628704 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.628719 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.636036 (+ 7317us) server_negotiation.cc:407] Connection header received
0807 18:26:19.636377 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.636386 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.636490 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.636573 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:19.637290 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.638433 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.645151 (+ 6718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.645783 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.645888 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.647795 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:19.647809 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:19.647828 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:19.648666 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:19.648733 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:19.648761 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:19.648863 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.649228 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.649356 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:26:19.651012 23275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:19.651297 22993 catalog_manager.cc:4841] TS 926aaf8b915642b6a462c2262d227d3c (127.29.156.194:42309): tablet 1c2928f160944de79f3b8d89cd4c8a1a (table test-table [id=6f300ae38d0e4e18b16f1e9bedfd8554]) successfully deleted
I20260807 18:26:19.748083 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 23102
I20260807 18:26:19.776810 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 23292
I20260807 18:26:19.785192 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.777675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.29.156.194:42309)
0807 18:26:19.777870 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.777876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.777898 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:26:19.778052 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.778058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.778183 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.778298 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:19.778744 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.780014 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.781365 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.782090 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.782231 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.783356 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:19.783367 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:19.783387 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:19.784290 (+ 903us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:19.784356 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:19.784381 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:19.784503 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.784657 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.784774 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:26:19.786044 23470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:19.907821 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 23292
I20260807 18:26:19.933037 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 23483
I20260807 18:26:19.940670 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:19.933889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.29.156.195:33989)
0807 18:26:19.934083 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:19.934091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:19.934194 (+ 103us) server_negotiation.cc:407] Connection header received
0807 18:26:19.934396 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:19.934400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:19.934480 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:19.934586 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:19.935013 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.935844 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.937158 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:19.937544 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:19.937686 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:19.939004 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:19.939019 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:19.939035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:19.939826 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:19.939879 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:19.939901 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:19.940083 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:19.940224 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:26:19.940346 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:26:19.941498 23656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:20.033632 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 23483
I20260807 18:26:20.055084 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 22973
I20260807 18:26:20.056315 23055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:20.170275 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 22973
2026-08-07T18:26:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3094 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:20 dist-test-slave-sb2z krb5kdc[23747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:20 dist-test-slave-sb2z krb5kdc[23747](info): set up 2 sockets
Aug 07 18:26:20 dist-test-slave-sb2z krb5kdc[23747](info): commencing operation
krb5kdc: starting...
W20260807 18:26:21.498983 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:21 dist-test-slave-sb2z krb5kdc[23747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:26:21.855082 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:38545
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:22.218544 23767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:22.218947 23767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:22.219118 23767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:22.226691 23767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:22.226876 23767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:22.226994 23767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:22.227108 23767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:22.227192 23767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:22.237572 23767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:22.239038 23767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:22.240487 23767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:22.253140 23779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:22.253150 23776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:22.253150 23777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:22.254652 23767 server_base.cc:1034] running on GCE node
I20260807 18:26:22.255719 23767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:22.257256 23767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:22.258530 23767 hybrid_clock.cc:648] HybridClock initialized: now 1786127182258495 us; error 94 us; skew 500 ppm
I20260807 18:26:22.258958 23767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:22 dist-test-slave-sb2z krb5kdc[23747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127182, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:22.266927 23767 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:26:22.268700 23767 webserver.cc:460] Webserver started at http://127.29.156.254:40577/ using document root <none> and password file <none>
I20260807 18:26:22.269587 23767 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:22.269737 23767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:22.270071 23767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:22.273433 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "dae4529ab19e40e0b109d87c97d560d8"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "16d8a59eebdeff0e9a5fe9840602b2a3"
server_key_iv: "17e24a6810451ecc97d00c7eefdb94e5"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.274248 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "dae4529ab19e40e0b109d87c97d560d8"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "16d8a59eebdeff0e9a5fe9840602b2a3"
server_key_iv: "17e24a6810451ecc97d00c7eefdb94e5"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.279855 23767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:26:22.284217 23790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:22.285764 23767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:26:22.285976 23767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "dae4529ab19e40e0b109d87c97d560d8"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "16d8a59eebdeff0e9a5fe9840602b2a3"
server_key_iv: "17e24a6810451ecc97d00c7eefdb94e5"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.286214 23767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:22.309252 23767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:22.313655 23767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:22.313957 23767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:22.334036 23767 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:38545
I20260807 18:26:22.334079 23866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:38545 every 8 connection(s)
I20260807 18:26:22.335448 23767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:22.339277 23867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:22.340992 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 23767
I20260807 18:26:22.341300 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:22.341773 30323 external_mini_cluster.cc:1428] Setting key 3cf28fb4c1f4d524b075c3ae2c289889
I20260807 18:26:22.351941 23867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: Bootstrap starting.
I20260807 18:26:22.356031 23867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:22.357452 23867 log.cc:824] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:26:22 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:22.361296 23867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: No bootstrap required, opened a new log
I20260807 18:26:22.368870 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:22.343830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.29.156.254:38545)
0807 18:26:22.344753 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:22.344765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:22.344811 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:26:22.346557 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:22.346581 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:22.346978 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:22.347294 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:22.349398 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:22.350930 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:22.352857 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:22.353581 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:22.353722 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:22.359352 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:22.359363 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:22.359383 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:22.359433 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:22.363726 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:22.364575 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:22.364588 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:22.364601 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:22.364834 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:22.365296 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:22.365301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:22.365304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:22.366085 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:22.366387 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:22.366826 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 18:26:22.367328 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":445,"threads_started":1}
I20260807 18:26:22.372219 23867 raft_consensus.cc:348] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } }
I20260807 18:26:22.372572 23867 raft_consensus.cc:374] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:22.372649 23867 raft_consensus.cc:729] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae4529ab19e40e0b109d87c97d560d8, State: Initialized, Role: FOLLOWER
I20260807 18:26:22.373147 23867 consensus_queue.cc:260] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } }
I20260807 18:26:22.373355 23867 raft_consensus.cc:388] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:22.373451 23867 raft_consensus.cc:482] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:22.373570 23867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:22.375772 23867 raft_consensus.cc:504] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } }
I20260807 18:26:22.376374 23867 leader_election.cc:302] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dae4529ab19e40e0b109d87c97d560d8; no voters:
I20260807 18:26:22.377166 23867 leader_election.cc:288] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:22.377305 23875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:22.378058 23875 raft_consensus.cc:686] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [term 1 LEADER]: Becoming Leader. State: Replica: dae4529ab19e40e0b109d87c97d560d8, State: Running, Role: LEADER
I20260807 18:26:22.378657 23875 consensus_queue.cc:237] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } }
I20260807 18:26:22.379271 23867 sys_catalog.cc:564] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:22.381536 23876 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dae4529ab19e40e0b109d87c97d560d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } } }
I20260807 18:26:22.382596 23876 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:22.382148 23877 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dae4529ab19e40e0b109d87c97d560d8. Latest consensus state: current_term: 1 leader_uuid: "dae4529ab19e40e0b109d87c97d560d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae4529ab19e40e0b109d87c97d560d8" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 38545 } } }
I20260807 18:26:22.385303 23877 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:22.391985 23887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:22.397992 23887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:22.408747 23887 catalog_manager.cc:1347] Generated new cluster ID: 5648f7d2a97149b6ad6a31383ef6a008
I20260807 18:26:22.408888 23887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:22.429070 23887 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:22.430469 23887 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:22.438035 23887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: Generated new TSK 0
I20260807 18:26:22.439183 23887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:26:22.520558 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:22.883468 23906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:22.883889 23906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:22.884001 23906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:22.892026 23906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:22.892161 23906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:22.892280 23906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:22.903054 23906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:22.904572 23906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:22.905934 23906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:22.920167 23912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:22.920256 23913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:22.920235 23916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:22.921350 23906 server_base.cc:1034] running on GCE node
I20260807 18:26:22.922119 23906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:22.923398 23906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:22.924687 23906 hybrid_clock.cc:648] HybridClock initialized: now 1786127182924659 us; error 67 us; skew 500 ppm
I20260807 18:26:22.925094 23906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:22 dist-test-slave-sb2z krb5kdc[23747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127182, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:22.928956 23906 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:26:22.930788 23906 webserver.cc:460] Webserver started at http://127.29.156.193:45703/ using document root <none> and password file <none>
I20260807 18:26:22.931679 23906 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:22.931861 23906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:22.932250 23906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:22.935662 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "76bf88c375754ab7a6529d526d04f8a1"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "6dc6ddb090d3cfebaf3d4767692714d2"
server_key_iv: "bb1e0ca9273ae7286b65785b547d99e8"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.936575 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "76bf88c375754ab7a6529d526d04f8a1"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "6dc6ddb090d3cfebaf3d4767692714d2"
server_key_iv: "bb1e0ca9273ae7286b65785b547d99e8"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.942596 23906 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:26:22.946599 23927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:22.948283 23906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:26:22.948478 23906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "76bf88c375754ab7a6529d526d04f8a1"
format_stamp: "Formatted at 2026-08-07 18:26:22 on dist-test-slave-sb2z"
server_key: "6dc6ddb090d3cfebaf3d4767692714d2"
server_key_iv: "bb1e0ca9273ae7286b65785b547d99e8"
server_key_version: "encryptionkey@0"
I20260807 18:26:22.948709 23906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:22.963053 23906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:22.968079 23906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:22.968417 23906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:22.969460 23906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:22.970897 23906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:22.971011 23906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:22.971177 23906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:22.971254 23906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:23.010854 23906 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:44839
I20260807 18:26:23.010902 24085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:44839 every 8 connection(s)
I20260807 18:26:23.012313 23906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:23.016626 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 23906
I20260807 18:26:23.016866 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:23.017511 30323 external_mini_cluster.cc:1428] Setting key 47ecf79abaf9e5c185176d4d430d3ef8
Aug 07 18:26:23 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127182, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:23.036996 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:23.015091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:51475 (local address 127.29.156.254:38545)
0807 18:26:23.015615 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:23.015622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:23.016890 (+ 1268us) server_negotiation.cc:407] Connection header received
0807 18:26:23.019450 (+ 2560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:23.019454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:23.019576 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:23.019702 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:23.021962 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.022870 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.025508 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.025991 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.026101 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:23.030598 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:23.030604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:23.030607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:23.030642 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:23.033369 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:23.034182 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:23.034187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:23.034189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:23.034317 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:23.034796 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:23.034808 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:23.034810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:23.035114 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:23.035475 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:23.036301 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 18:26:23.036553 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":226,"threads_started":1}
I20260807 18:26:23.037999 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:23.015667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:38545 (local address 127.29.156.193:51475)
0807 18:26:23.016677 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:23.016747 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:23.019086 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:23.020003 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:23.020016 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:23.020807 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:23.021690 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:23.021721 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.023074 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.023082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:23.025335 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:23.025355 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.026245 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.026255 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:23.026609 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:23.027380 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:23.027415 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:23.030331 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:23.033525 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:23.033534 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:23.033553 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:23.034039 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:23.034435 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:23.034441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:23.034445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:23.034666 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:23.035454 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:23.035464 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:23.035888 (+ 424us) client_negotiation.cc:769] Sending connection context
0807 18:26:23.036180 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:26:23.036769 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":390,"threads_started":1}
I20260807 18:26:23.039916 24086 heartbeater.cc:347] Connected to a master server at 127.29.156.254:38545
I20260807 18:26:23.040340 24086 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:23.041392 24086 heartbeater.cc:511] Master 127.29.156.254:38545 requested a full tablet report, sending...
I20260807 18:26:23.044292 23815 ts_manager.cc:194] Registered new tserver with Master: 76bf88c375754ab7a6529d526d04f8a1 (127.29.156.193:44839)
I20260807 18:26:23.046867 23815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:51475
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:26:23.103377 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:23.465166 24095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:23.465463 24095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:23.465631 24095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:23.472981 24095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:23.473090 24095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:23.473275 24095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:23.483228 24095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:23.484614 24095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:23.485952 24095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:23.500325 24106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:23.500279 24103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:23.500643 24100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:23.501251 24095 server_base.cc:1034] running on GCE node
I20260807 18:26:23.501998 24095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:23.503223 24095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:23.504508 24095 hybrid_clock.cc:648] HybridClock initialized: now 1786127183504424 us; error 79 us; skew 500 ppm
I20260807 18:26:23.504875 24095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:23 dist-test-slave-sb2z krb5kdc[23747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:23.508589 24095 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:26:23.510324 24095 webserver.cc:460] Webserver started at http://127.29.156.194:33831/ using document root <none> and password file <none>
I20260807 18:26:23.511132 24095 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:23.511253 24095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:23.511727 24095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:23.515292 24095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "bfc00cc248624f069f05974ce79dfafd"
format_stamp: "Formatted at 2026-08-07 18:26:23 on dist-test-slave-sb2z"
server_key: "5f4b9b1006cb3874c6c15b94b609f87a"
server_key_iv: "8e38970cffdca1333e34a46cac77d413"
server_key_version: "encryptionkey@0"
I20260807 18:26:23.516323 24095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "bfc00cc248624f069f05974ce79dfafd"
format_stamp: "Formatted at 2026-08-07 18:26:23 on dist-test-slave-sb2z"
server_key: "5f4b9b1006cb3874c6c15b94b609f87a"
server_key_iv: "8e38970cffdca1333e34a46cac77d413"
server_key_version: "encryptionkey@0"
I20260807 18:26:23.522970 24095 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:26:23.527107 24115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:23.528961 24095 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:26:23.529165 24095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "bfc00cc248624f069f05974ce79dfafd"
format_stamp: "Formatted at 2026-08-07 18:26:23 on dist-test-slave-sb2z"
server_key: "5f4b9b1006cb3874c6c15b94b609f87a"
server_key_iv: "8e38970cffdca1333e34a46cac77d413"
server_key_version: "encryptionkey@0"
I20260807 18:26:23.529443 24095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:23.548583 24095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:23.553112 24095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:23.553457 24095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:23.554579 24095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:23.556205 24095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:23.556289 24095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:23.556453 24095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:23.556514 24095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:23.594075 24095 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:46365
I20260807 18:26:23.594190 24278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:46365 every 8 connection(s)
I20260807 18:26:23.595594 24095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:23.600028 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 24095
I20260807 18:26:23.600265 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:23.600937 30323 external_mini_cluster.cc:1428] Setting key 7561b13a2ce1125eeceb71be9c23d250
Aug 07 18:26:23 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:23.621568 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:23.598814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:38293 (local address 127.29.156.254:38545)
0807 18:26:23.599325 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:23.599331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:23.600570 (+ 1239us) server_negotiation.cc:407] Connection header received
0807 18:26:23.603014 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:23.603022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:23.603108 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:23.603230 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:23.605596 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.606362 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.608989 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.609452 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.609572 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:23.614964 (+ 5392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:23.614974 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:23.614980 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:23.615032 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:23.618094 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:23.618894 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:23.618900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:23.618904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:23.619043 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:23.619548 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:23.619556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:23.619561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:23.619868 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:23.620041 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:23.620882 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 18:26:23.621149 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":284,"threads_started":1}
I20260807 18:26:23.622596 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:23.599281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:38545 (local address 127.29.156.194:38293)
0807 18:26:23.600344 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:23.600408 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:23.602669 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:23.603566 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:23.603579 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:23.604486 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:23.605365 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:23.605397 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.606589 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.606597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:23.608801 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:23.608820 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:23.609729 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:23.609740 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:23.610203 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:23.611113 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:23.611148 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:23.614564 (+ 3416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:23.618263 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:23.618273 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:23.618292 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:23.618747 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:23.619171 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:23.619176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:23.619180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:23.619391 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:23.620013 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:23.620022 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:23.620486 (+ 464us) client_negotiation.cc:769] Sending connection context
0807 18:26:23.620783 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 18:26:23.621366 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":407,"threads_started":1}
I20260807 18:26:23.624221 24280 heartbeater.cc:347] Connected to a master server at 127.29.156.254:38545
I20260807 18:26:23.624575 24280 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:23.625368 24280 heartbeater.cc:511] Master 127.29.156.254:38545 requested a full tablet report, sending...
I20260807 18:26:23.627050 23815 ts_manager.cc:194] Registered new tserver with Master: bfc00cc248624f069f05974ce79dfafd (127.29.156.194:46365)
I20260807 18:26:23.628398 23815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:38293
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:26:23.681218 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:24.039901 24289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:24.040311 24289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:24.040488 24289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:24.048125 24289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:24.048280 24289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:24.048472 24289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:24.051317 24086 heartbeater.cc:503] Master 127.29.156.254:38545 was elected leader, sending a full tablet report...
I20260807 18:26:24.059700 24289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:24.061110 24289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:24.062420 24289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:24.078191 24296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:24.078235 24294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:24.078151 24302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:24.079130 24289 server_base.cc:1034] running on GCE node
I20260807 18:26:24.080029 24289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:24.081209 24289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:24.082448 24289 hybrid_clock.cc:648] HybridClock initialized: now 1786127184082389 us; error 76 us; skew 500 ppm
I20260807 18:26:24.082785 24289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:24 dist-test-slave-sb2z krb5kdc[23747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127184, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:24.086448 24289 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:26:24.088163 24289 webserver.cc:460] Webserver started at http://127.29.156.195:42619/ using document root <none> and password file <none>
I20260807 18:26:24.088914 24289 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:24.089076 24289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:24.089473 24289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:24.092947 24289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "06ba19acd09b4fa0a916950e677bbdc4"
format_stamp: "Formatted at 2026-08-07 18:26:24 on dist-test-slave-sb2z"
server_key: "17c1a2a09f9cb8ae3839164853620423"
server_key_iv: "a09ba1744d90328300521e73e5af99d8"
server_key_version: "encryptionkey@0"
I20260807 18:26:24.093797 24289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "06ba19acd09b4fa0a916950e677bbdc4"
format_stamp: "Formatted at 2026-08-07 18:26:24 on dist-test-slave-sb2z"
server_key: "17c1a2a09f9cb8ae3839164853620423"
server_key_iv: "a09ba1744d90328300521e73e5af99d8"
server_key_version: "encryptionkey@0"
I20260807 18:26:24.100057 24289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260807 18:26:24.104264 24313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:24.106040 24289 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:26:24.106272 24289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "06ba19acd09b4fa0a916950e677bbdc4"
format_stamp: "Formatted at 2026-08-07 18:26:24 on dist-test-slave-sb2z"
server_key: "17c1a2a09f9cb8ae3839164853620423"
server_key_iv: "a09ba1744d90328300521e73e5af99d8"
server_key_version: "encryptionkey@0"
I20260807 18:26:24.106528 24289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:24.125294 24289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:24.129709 24289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:24.130023 24289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:24.131080 24289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:24.132616 24289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:24.132727 24289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:24.132885 24289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:24.132959 24289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:24.172544 24289 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33485
I20260807 18:26:24.172598 24474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33485 every 8 connection(s)
I20260807 18:26:24.173981 24289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:24.177229 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 24289
I20260807 18:26:24.177470 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:24.177980 30323 external_mini_cluster.cc:1428] Setting key 3deb888ab5b6928412133c6279482e09
Aug 07 18:26:24 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127184, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:24.197062 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.176880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43069 (local address 127.29.156.254:38545)
0807 18:26:24.177489 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.177496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.178606 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 18:26:24.181189 (+ 2583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.181194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.181306 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.181429 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:24.183008 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.183869 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.185496 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.185898 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.186037 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.189872 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:24.189878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:24.189881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:24.189915 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:24.193637 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.194535 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.194542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.194546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.194684 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.195156 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.195162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.195165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.195445 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:24.195647 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.196400 (+ 753us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.196651 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":303,"threads_started":1}
I20260807 18:26:24.198050 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.177489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:38545 (local address 127.29.156.195:43069)
0807 18:26:24.178408 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.178457 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.180834 (+ 2377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.181690 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.181699 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.182267 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:24.182831 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.182848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.184043 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.184048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.185339 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.185353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.186146 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.186155 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.186395 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.187114 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:24.187138 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:24.189617 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:24.193790 (+ 4173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:24.193800 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:24.193817 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:24.194404 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:24.194805 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:24.194810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:24.194814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:24.195036 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:24.195628 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:24.195637 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:24.196029 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.196319 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.196885 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":374,"threads_started":1}
I20260807 18:26:24.199673 24476 heartbeater.cc:347] Connected to a master server at 127.29.156.254:38545
I20260807 18:26:24.199999 24476 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:24.200763 24476 heartbeater.cc:511] Master 127.29.156.254:38545 requested a full tablet report, sending...
I20260807 18:26:24.202226 23815 ts_manager.cc:194] Registered new tserver with Master: 06ba19acd09b4fa0a916950e677bbdc4 (127.29.156.195:33485)
I20260807 18:26:24.203038 23815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:43069
I20260807 18:26:24.206737 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:24.226469 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.211639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.29.156.254:38545)
0807 18:26:24.211824 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.211829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.212362 (+ 533us) server_negotiation.cc:407] Connection header received
0807 18:26:24.212477 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.212481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.212545 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.212633 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:24.214126 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.214871 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.216504 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.216931 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.217048 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.220128 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:24.220133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:24.220137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:24.220172 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:24.223034 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.224050 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.224054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.224057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.224264 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.224831 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.224835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.224837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.225192 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:24.225406 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.226058 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.226240 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:26:24.230981 23815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:24.238257 23815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:24.264233 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.251423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46365 (local address 127.0.0.1:44200)
0807 18:26:24.253601 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.253623 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.253742 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.254782 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.254793 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.254837 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.255379 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.255400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.257433 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.257440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.260647 (+ 3207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.260662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.262975 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.262983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.263216 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.263275 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.263444 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.263705 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2035,"mutex_wait_us":680,"thread_start_us":1227,"threads_started":1}
I20260807 18:26:24.264984 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.251349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33485 (local address 127.0.0.1:48748)
0807 18:26:24.252193 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.252252 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.252426 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.254919 (+ 2493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.254924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.254954 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.255444 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.255454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.259101 (+ 3647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.259108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.261366 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.261382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.264005 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.264011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.264242 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.264298 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.264371 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.264509 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":1,"thread_start_us":2014,"threads_started":1}
I20260807 18:26:24.265445 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.251987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.29.156.194:46365)
0807 18:26:24.253737 (+ 1750us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.253748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.253781 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:26:24.253995 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.254006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.254277 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.254491 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.255630 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.257236 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.260850 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.262463 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.262643 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.262836 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.264882 (+ 2046us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.265098 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1504,"thread_start_us":2011,"threads_started":1}
I20260807 18:26:24.266731 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.251712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.29.156.195:33485)
0807 18:26:24.254302 (+ 2590us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.254314 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.254361 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:26:24.254463 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.254470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.254699 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.256983 (+ 2284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.257279 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.258924 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.262301 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.263763 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.265256 (+ 1493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.266121 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.266204 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.266379 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2412,"thread_start_us":2742,"threads_started":1}
I20260807 18:26:24.270056 24177 tablet_service.cc:1467] Processing CreateTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 (DEFAULT_TABLE table=test-table [id=421d85f748634e7d90c49318ab0f22f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:24.271867 24177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a6fbd5d8e640349dd06f8dcdb07474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:24.271509 24379 tablet_service.cc:1467] Processing CreateTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 (DEFAULT_TABLE table=test-table [id=421d85f748634e7d90c49318ab0f22f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:24.273000 24379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a6fbd5d8e640349dd06f8dcdb07474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:24.275027 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.253861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:44839 (local address 127.0.0.1:46276)
0807 18:26:24.266911 (+ 13050us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.266933 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.267068 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.267653 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.267658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.267685 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.268069 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.268084 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.270089 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.270097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.272154 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.272171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.274114 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.274121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.274297 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.274332 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.274445 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.274621 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11578,"thread_start_us":10774,"threads_started":1}
I20260807 18:26:24.275079 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.254194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.29.156.193:44839)
0807 18:26:24.266590 (+ 12396us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.266599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.267069 (+ 470us) server_negotiation.cc:407] Connection header received
0807 18:26:24.267212 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.267218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.267421 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.267673 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.268270 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.269887 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.272364 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.273865 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.274109 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.274284 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.274510 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.274696 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12189,"thread_start_us":12401,"threads_started":1}
I20260807 18:26:24.279512 23992 tablet_service.cc:1467] Processing CreateTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 (DEFAULT_TABLE table=test-table [id=421d85f748634e7d90c49318ab0f22f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:24.281189 23992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a6fbd5d8e640349dd06f8dcdb07474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:24.286760 24505 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Bootstrap starting.
I20260807 18:26:24.288218 24506 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Bootstrap starting.
I20260807 18:26:24.290979 24505 tablet_bootstrap.cc:654] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:24.292033 24506 tablet_bootstrap.cc:654] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:24.292438 24505 log.cc:824] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:24.293350 24506 log.cc:824] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:24.295894 24505 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: No bootstrap required, opened a new log
I20260807 18:26:24.296214 24505 ts_tablet_manager.cc:1397] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:26:24.297492 24510 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Bootstrap starting.
I20260807 18:26:24.298193 24506 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: No bootstrap required, opened a new log
I20260807 18:26:24.298556 24506 ts_tablet_manager.cc:1397] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 18:26:24.301239 24510 tablet_bootstrap.cc:654] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:24.302554 24510 log.cc:824] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:24.304774 24505 raft_consensus.cc:348] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.305253 24505 raft_consensus.cc:374] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:24.305368 24505 raft_consensus.cc:729] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ba19acd09b4fa0a916950e677bbdc4, State: Initialized, Role: FOLLOWER
I20260807 18:26:24.305927 24510 tablet_bootstrap.cc:492] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: No bootstrap required, opened a new log
I20260807 18:26:24.306196 24510 ts_tablet_manager.cc:1397] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:26:24.306002 24505 consensus_queue.cc:260] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.307699 24476 heartbeater.cc:503] Master 127.29.156.254:38545 was elected leader, sending a full tablet report...
I20260807 18:26:24.308950 24505 ts_tablet_manager.cc:1428] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260807 18:26:24.309588 24506 raft_consensus.cc:348] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.310032 24506 raft_consensus.cc:374] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:24.310158 24506 raft_consensus.cc:729] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc00cc248624f069f05974ce79dfafd, State: Initialized, Role: FOLLOWER
I20260807 18:26:24.310828 24506 consensus_queue.cc:260] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.316260 24280 heartbeater.cc:503] Master 127.29.156.254:38545 was elected leader, sending a full tablet report...
I20260807 18:26:24.317181 24506 ts_tablet_manager.cc:1428] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Time spent starting tablet: real 0.019s user 0.011s sys 0.007s
I20260807 18:26:24.318035 24510 raft_consensus.cc:348] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.318496 24510 raft_consensus.cc:374] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:24.318621 24510 raft_consensus.cc:729] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bf88c375754ab7a6529d526d04f8a1, State: Initialized, Role: FOLLOWER
I20260807 18:26:24.319262 24510 consensus_queue.cc:260] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.321069 24510 ts_tablet_manager.cc:1428] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260807 18:26:24.348026 24281 tablet.cc:2369] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:24.425884 24477 tablet.cc:2369] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:24.515316 24087 tablet.cc:2369] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:24.686816 24516 raft_consensus.cc:482] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:24.687311 24516 raft_consensus.cc:504] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.689213 24516 leader_election.cc:288] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ba19acd09b4fa0a916950e677bbdc4 (127.29.156.195:33485), bfc00cc248624f069f05974ce79dfafd (127.29.156.194:46365)
I20260807 18:26:24.693976 24512 raft_consensus.cc:482] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:24.694315 24512 raft_consensus.cc:504] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.696311 24512 leader_election.cc:288] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfc00cc248624f069f05974ce79dfafd (127.29.156.194:46365), 76bf88c375754ab7a6529d526d04f8a1 (127.29.156.193:44839)
I20260807 18:26:24.697731 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.689843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:44933 (local address 127.29.156.195:33485)
0807 18:26:24.690206 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.690216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.690710 (+ 494us) server_negotiation.cc:407] Connection header received
0807 18:26:24.690964 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.690970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.691077 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.691240 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.692025 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.692888 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.695190 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.696488 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.696666 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.696761 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.697335 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.697473 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260807 18:26:24.697995 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.689748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:38909 (local address 127.29.156.194:46365)
0807 18:26:24.690081 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.690090 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.690847 (+ 757us) server_negotiation.cc:407] Connection header received
0807 18:26:24.691074 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.691113 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.691202 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.691306 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.692145 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.692912 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.695546 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.696835 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.697002 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.697078 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.697587 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.697733 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260807 18:26:24.698453 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.689690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33485 (local address 127.29.156.193:44933)
0807 18:26:24.690548 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.690576 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.690757 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.691257 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.691262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.691291 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.691844 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.691858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.693028 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.693035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.695014 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.695028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.696923 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.696933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.697124 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.697171 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.697894 (+ 723us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.698043 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":110,"thread_start_us":423,"threads_started":1}
I20260807 18:26:24.699260 24518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.689752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46365 (local address 127.29.156.193:38909)
0807 18:26:24.690711 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.690728 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.690885 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.691323 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.691328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.691361 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.691944 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.691959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.693056 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.693061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.695338 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.695354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.697235 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.697244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.697406 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.697437 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.698719 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.698855 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":158,"thread_start_us":554,"threads_started":1}
I20260807 18:26:24.699723 24412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "76bf88c375754ab7a6529d526d04f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ba19acd09b4fa0a916950e677bbdc4" is_pre_election: true
I20260807 18:26:24.700273 24412 raft_consensus.cc:2454] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bf88c375754ab7a6529d526d04f8a1 in term 0.
I20260807 18:26:24.701064 23930 leader_election.cc:302] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ba19acd09b4fa0a916950e677bbdc4, 76bf88c375754ab7a6529d526d04f8a1; no voters:
I20260807 18:26:24.701267 24217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "76bf88c375754ab7a6529d526d04f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc00cc248624f069f05974ce79dfafd" is_pre_election: true
I20260807 18:26:24.701558 24516 raft_consensus.cc:2781] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:24.701726 24516 raft_consensus.cc:482] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:24.701833 24516 raft_consensus.cc:3037] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:24.701858 24217 raft_consensus.cc:2454] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bf88c375754ab7a6529d526d04f8a1 in term 0.
I20260807 18:26:24.704573 24516 raft_consensus.cc:504] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.705176 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.696645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:46365 (local address 127.29.156.195:33605)
0807 18:26:24.697063 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.697085 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.697192 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.698639 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.698644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.698673 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.698849 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.698858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.700530 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.700536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.702739 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.702750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.704366 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.704373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.704624 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.704662 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.704778 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.704916 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260807 18:26:24.705308 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.697392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:33605 (local address 127.29.156.194:46365)
0807 18:26:24.698286 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.698291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.698311 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:26:24.698366 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.698370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.698465 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.698997 (+ 532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.699136 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.700384 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.702920 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.704181 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.704349 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.704419 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.704830 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.704979 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":2970,"threads_started":1}
I20260807 18:26:24.705916 24412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "76bf88c375754ab7a6529d526d04f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ba19acd09b4fa0a916950e677bbdc4"
I20260807 18:26:24.705960 24217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "06ba19acd09b4fa0a916950e677bbdc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc00cc248624f069f05974ce79dfafd" is_pre_election: true
I20260807 18:26:24.706144 24412 raft_consensus.cc:3037] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:24.706200 24217 raft_consensus.cc:2454] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ba19acd09b4fa0a916950e677bbdc4 in term 0.
I20260807 18:26:24.706506 24215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "76bf88c375754ab7a6529d526d04f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc00cc248624f069f05974ce79dfafd"
I20260807 18:26:24.706765 24215 raft_consensus.cc:3037] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:24.706818 24318 leader_election.cc:302] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ba19acd09b4fa0a916950e677bbdc4, bfc00cc248624f069f05974ce79dfafd; no voters:
I20260807 18:26:24.707469 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.697580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:43307 (local address 127.29.156.193:44839)
0807 18:26:24.697815 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.697823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.700104 (+ 2281us) server_negotiation.cc:407] Connection header received
0807 18:26:24.700346 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.700351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.700427 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.700491 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:24.701376 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.702471 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.704960 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.706283 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.706449 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.706520 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.707126 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.707262 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:26:24.708133 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.696808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:44839 (local address 127.29.156.195:43307)
0807 18:26:24.700006 (+ 3198us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:24.700028 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:24.700208 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:24.700691 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:24.700701 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:24.700726 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:24.701158 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.701170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.702688 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.702695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.704763 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:24.704780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.706682 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.706688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:24.706907 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.706952 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:26:24.707629 (+ 677us) client_negotiation.cc:240] Negotiation successful
0807 18:26:24.707769 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3074,"mutex_wait_us":79,"thread_start_us":2838,"threads_started":1}
I20260807 18:26:24.708623 24516 leader_election.cc:288] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 06ba19acd09b4fa0a916950e677bbdc4 (127.29.156.195:33485), bfc00cc248624f069f05974ce79dfafd (127.29.156.194:46365)
I20260807 18:26:24.708973 24020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a6fbd5d8e640349dd06f8dcdb07474" candidate_uuid: "06ba19acd09b4fa0a916950e677bbdc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76bf88c375754ab7a6529d526d04f8a1" is_pre_election: true
I20260807 18:26:24.709467 24020 raft_consensus.cc:2379] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ba19acd09b4fa0a916950e677bbdc4 in current term 1: Already voted for candidate 76bf88c375754ab7a6529d526d04f8a1 in this term.
I20260807 18:26:24.710376 24215 raft_consensus.cc:2454] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bf88c375754ab7a6529d526d04f8a1 in term 1.
I20260807 18:26:24.710676 24412 raft_consensus.cc:2454] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bf88c375754ab7a6529d526d04f8a1 in term 1.
I20260807 18:26:24.710795 23931 leader_election.cc:302] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76bf88c375754ab7a6529d526d04f8a1, bfc00cc248624f069f05974ce79dfafd; no voters:
I20260807 18:26:24.710888 24512 raft_consensus.cc:2741] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 18:26:24.711228 24516 raft_consensus.cc:2781] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:24.711817 24516 raft_consensus.cc:686] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 76bf88c375754ab7a6529d526d04f8a1, State: Running, Role: LEADER
I20260807 18:26:24.712373 24516 consensus_queue.cc:237] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } }
I20260807 18:26:24.717921 23815 catalog_manager.cc:5495] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76bf88c375754ab7a6529d526d04f8a1 (127.29.156.193). New cstate: current_term: 1 leader_uuid: "76bf88c375754ab7a6529d526d04f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76bf88c375754ab7a6529d526d04f8a1" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 44839 } health_report { overall_health: HEALTHY } } }
I20260807 18:26:24.838294 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.831128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.29.156.193:44839)
0807 18:26:24.831446 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.831455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.832000 (+ 545us) server_negotiation.cc:407] Connection header received
0807 18:26:24.832240 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.832244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.832349 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.832527 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:24.833069 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.833950 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.835718 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.836192 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.836358 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.837038 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:24.837576 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:24.837780 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.837961 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.838017 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:26:24.846820 24412 raft_consensus.cc:1261] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 1 FOLLOWER]: Refusing update from remote peer 76bf88c375754ab7a6529d526d04f8a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:24.846850 24215 raft_consensus.cc:1261] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 1 FOLLOWER]: Refusing update from remote peer 76bf88c375754ab7a6529d526d04f8a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:24.847801 24523 consensus_queue.cc:1035] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ba19acd09b4fa0a916950e677bbdc4" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:24.848166 24516 consensus_queue.cc:1035] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc00cc248624f069f05974ce79dfafd" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 46365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:24.877103 23815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:24.877385 23815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:24.881285 23815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dae4529ab19e40e0b109d87c97d560d8: Sending DeleteTablet for 3 replicas of tablet e6a6fbd5d8e640349dd06f8dcdb07474
I20260807 18:26:24.882692 24177 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:24 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:44200
I20260807 18:26:24.882747 23992 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:24 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:46276
I20260807 18:26:24.882756 24379 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a6fbd5d8e640349dd06f8dcdb07474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:24 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:48748
I20260807 18:26:24.883726 24542 tablet_replica.cc:330] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: stopping tablet replica
I20260807 18:26:24.884354 24544 tablet_replica.cc:330] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: stopping tablet replica
I20260807 18:26:24.884991 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 23906
I20260807 18:26:24.885183 24544 raft_consensus.cc:2229] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:24.885809 24544 raft_consensus.cc:2258] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:24.888957 24542 raft_consensus.cc:2229] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:24.889770 24542 raft_consensus.cc:2258] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:24.892750 24542 ts_tablet_manager.cc:1905] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:24.894194 24543 tablet_replica.cc:330] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: stopping tablet replica
I20260807 18:26:24.894744 24543 raft_consensus.cc:2229] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:24.895347 24544 ts_tablet_manager.cc:1905] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:24.895635 24543 raft_consensus.cc:2258] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:24.897828 24543 ts_tablet_manager.cc:1905] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:24.901736 24542 ts_tablet_manager.cc:1918] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:24.901983 24542 log.cc:1196] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/e6a6fbd5d8e640349dd06f8dcdb07474
I20260807 18:26:24.902591 24542 ts_tablet_manager.cc:1939] T e6a6fbd5d8e640349dd06f8dcdb07474 P bfc00cc248624f069f05974ce79dfafd: Deleting consensus metadata
I20260807 18:26:24.904825 23795 catalog_manager.cc:4841] TS bfc00cc248624f069f05974ce79dfafd (127.29.156.194:46365): tablet e6a6fbd5d8e640349dd06f8dcdb07474 (table test-table [id=421d85f748634e7d90c49318ab0f22f3]) successfully deleted
I20260807 18:26:24.907485 24543 ts_tablet_manager.cc:1918] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:24.907809 24543 log.cc:1196] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/e6a6fbd5d8e640349dd06f8dcdb07474
I20260807 18:26:24.908499 24543 ts_tablet_manager.cc:1939] T e6a6fbd5d8e640349dd06f8dcdb07474 P 76bf88c375754ab7a6529d526d04f8a1: Deleting consensus metadata
I20260807 18:26:24.909292 24544 ts_tablet_manager.cc:1918] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:24.909525 24544 log.cc:1196] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/e6a6fbd5d8e640349dd06f8dcdb07474
I20260807 18:26:24.910151 24544 ts_tablet_manager.cc:1939] T e6a6fbd5d8e640349dd06f8dcdb07474 P 06ba19acd09b4fa0a916950e677bbdc4: Deleting consensus metadata
Aug 07 18:26:24 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:26:24.910661 23794 catalog_manager.cc:4841] TS 76bf88c375754ab7a6529d526d04f8a1 (127.29.156.193:44839): tablet e6a6fbd5d8e640349dd06f8dcdb07474 (table test-table [id=421d85f748634e7d90c49318ab0f22f3]) successfully deleted
I20260807 18:26:24.912132 23794 catalog_manager.cc:4841] TS 06ba19acd09b4fa0a916950e677bbdc4 (127.29.156.195:33485): tablet e6a6fbd5d8e640349dd06f8dcdb07474 (table test-table [id=421d85f748634e7d90c49318ab0f22f3]) successfully deleted
I20260807 18:26:24.918763 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:24.886350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46290 (local address 127.29.156.193:44839)
0807 18:26:24.886556 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:24.886563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:24.893401 (+ 6838us) server_negotiation.cc:407] Connection header received
0807 18:26:24.893685 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:24.893693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:24.893798 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:24.893934 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:24.899828 (+ 5894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.901354 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.904683 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:24.905464 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:24.905584 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:24.911741 (+ 6157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:24.911758 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:24.911778 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:24.911831 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:24.915248 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.916270 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.916277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.916279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.916421 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:24.916953 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:24.916957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:24.916960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:24.917353 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:24.917613 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:24.918158 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 18:26:24.918376 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:26:24.919588 24080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:25.022585 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 23906
I20260807 18:26:25.053673 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 24095
Aug 07 18:26:25 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:26:25.069762 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:25.054688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.29.156.194:46365)
0807 18:26:25.054888 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:25.054893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:25.054921 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:26:25.055263 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:25.055269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:25.055364 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:25.055469 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:25.056991 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:25.057748 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:25.058996 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:25.059409 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:25.059505 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:25.064085 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:25.064102 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:25.064119 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:25.064154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:25.067102 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:25.067852 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:25.067859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:25.067862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:25.068032 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:25.068389 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:25.068393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:25.068395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:25.068749 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:25.069036 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:25.069246 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 18:26:25.069409 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:26:25.070551 24272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:25.162022 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 24095
I20260807 18:26:25.189896 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 24289
Aug 07 18:26:25 dist-test-slave-sb2z krb5kdc[23747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:26:25.207765 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:25.190851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.29.156.195:33485)
0807 18:26:25.191098 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:25.191103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:25.191122 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:26:25.191371 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:25.191376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:25.191441 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:25.191592 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:25.193142 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:25.193907 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:25.195408 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:25.195883 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:25.195982 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:25.201104 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:25.201121 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:25.201138 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:25.201184 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:25.204712 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:25.205572 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:25.205582 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:25.205586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:25.205839 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:25.206240 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:25.206244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:25.206246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:25.206647 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:25.206915 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:25.207184 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:26:25.207365 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:26:25.208670 24466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:25.304693 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 24289
I20260807 18:26:25.334295 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 23767
I20260807 18:26:25.336225 23860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:25.578049 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 23767
2026-08-07T18:26:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5443 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:26:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:25Z Disabled control of system clock
I20260807 18:26:25.769142 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37713
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:26.200269 24557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:26.200587 24557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:26.200766 24557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:26.208247 24557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:26.208364 24557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:26.208460 24557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:26.208575 24557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:26.208657 24557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:26.218323 24557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:37713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:26.219743 24557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:26.221658 24557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:26.233943 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:26.234081 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:26.234086 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:26.235435 24557 server_base.cc:1034] running on GCE node
I20260807 18:26:26.236332 24557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:26.237818 24557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:26.239084 24557 hybrid_clock.cc:648] HybridClock initialized: now 1786127186239007 us; error 66 us; skew 500 ppm
I20260807 18:26:26.239462 24557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:26.241946 24557 webserver.cc:460] Webserver started at http://127.29.156.254:33651/ using document root <none> and password file <none>
I20260807 18:26:26.242661 24557 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:26.242794 24557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:26.243126 24557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:26.246070 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "19c00cc27d2042a4b0b8d9279fa13cfc"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "20c2c5376702354c7d963e55bee62f18"
server_key_iv: "8884ebbdbe177f2eb1f0877158e3fd6d"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.247049 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "19c00cc27d2042a4b0b8d9279fa13cfc"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "20c2c5376702354c7d963e55bee62f18"
server_key_iv: "8884ebbdbe177f2eb1f0877158e3fd6d"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.252588 24557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:26:26.256110 24583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:26.257494 24557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:26:26.257709 24557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "19c00cc27d2042a4b0b8d9279fa13cfc"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "20c2c5376702354c7d963e55bee62f18"
server_key_iv: "8884ebbdbe177f2eb1f0877158e3fd6d"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.257934 24557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:26.275674 24557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:26.276507 24557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:26.276810 24557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:26.296823 24557 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:37713
I20260807 18:26:26.296880 24649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:37713 every 8 connection(s)
I20260807 18:26:26.298189 24557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:26.301999 24650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:26.302196 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 24557
I20260807 18:26:26.302376 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:26.302838 30323 external_mini_cluster.cc:1428] Setting key 0ae8ef1d4d281f6657bc147f94cc0532
I20260807 18:26:26.316771 24650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: Bootstrap starting.
I20260807 18:26:26.319550 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:26.304925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.29.156.254:37713)
0807 18:26:26.306065 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:26.306080 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:26.306132 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:26:26.308993 (+ 2861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:26.309021 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:26.309634 (+ 613us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:26.310099 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:26.310617 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.312505 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.313984 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.315075 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.315248 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:26.316608 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:26.316619 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:26.316637 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:26.317368 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:26.317467 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:26.317495 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:26.317609 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:26.317852 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:26:26.318189 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":321,"threads_started":1}
I20260807 18:26:26.320740 24650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:26.321822 24650 log.cc:824] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:26.324676 24650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: No bootstrap required, opened a new log
I20260807 18:26:26.332394 24650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } }
I20260807 18:26:26.332768 24650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:26.332878 24650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c00cc27d2042a4b0b8d9279fa13cfc, State: Initialized, Role: FOLLOWER
I20260807 18:26:26.333483 24650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } }
I20260807 18:26:26.333798 24650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:26.333947 24650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:26.334149 24650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:26.336409 24650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } }
I20260807 18:26:26.336920 24650 leader_election.cc:302] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c00cc27d2042a4b0b8d9279fa13cfc; no voters:
I20260807 18:26:26.337532 24650 leader_election.cc:288] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:26.337776 24655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:26.338749 24655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [term 1 LEADER]: Becoming Leader. State: Replica: 19c00cc27d2042a4b0b8d9279fa13cfc, State: Running, Role: LEADER
I20260807 18:26:26.339304 24655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } }
I20260807 18:26:26.339864 24650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:26.342334 24659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c00cc27d2042a4b0b8d9279fa13cfc. Latest consensus state: current_term: 1 leader_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } } }
I20260807 18:26:26.342921 24659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:26.343127 24657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c00cc27d2042a4b0b8d9279fa13cfc" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 37713 } } }
I20260807 18:26:26.343482 24657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:26.347371 24666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:26.353173 24666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:26.364066 24666 catalog_manager.cc:1347] Generated new cluster ID: 254d8b9097984cfea01f1d256087da2e
I20260807 18:26:26.364197 24666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:26.396920 24666 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:26.398420 24666 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:26.410521 24666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: Generated new TSK 0
I20260807 18:26:26.411451 24666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:26.423859 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--builtin_ntp_servers=127.29.156.212:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:26.799034 24684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:26.799319 24684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:26.799408 24684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:26.806856 24684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:26.806967 24684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:26.807076 24684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:26.816349 24684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:26.817610 24684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:26.819437 24684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:26.833338 24693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:26.833281 24695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:26.833361 24697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:26.834038 24684 server_base.cc:1034] running on GCE node
I20260807 18:26:26.834692 24684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:26.835824 24684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:26.837093 24684 hybrid_clock.cc:648] HybridClock initialized: now 1786127186837041 us; error 68 us; skew 500 ppm
I20260807 18:26:26.837461 24684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:26.840490 24684 webserver.cc:460] Webserver started at http://127.29.156.193:33035/ using document root <none> and password file <none>
I20260807 18:26:26.841301 24684 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:26.841457 24684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:26.841804 24684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:26.844945 24684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "e951f25d25fb4e2aa3ae248a525edb38"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "af6ad60799b75e2a5e9b44b70d0631f8"
server_key_iv: "d9322a081ee90326ea5e83b33b6c71b5"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.845862 24684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "e951f25d25fb4e2aa3ae248a525edb38"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "af6ad60799b75e2a5e9b44b70d0631f8"
server_key_iv: "d9322a081ee90326ea5e83b33b6c71b5"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.851439 24684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:26:26.855304 24706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:26.856988 24684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:26:26.857228 24684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "e951f25d25fb4e2aa3ae248a525edb38"
format_stamp: "Formatted at 2026-08-07 18:26:26 on dist-test-slave-sb2z"
server_key: "af6ad60799b75e2a5e9b44b70d0631f8"
server_key_iv: "d9322a081ee90326ea5e83b33b6c71b5"
server_key_version: "encryptionkey@0"
I20260807 18:26:26.857483 24684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:26.887254 24684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:26.888284 24684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:26.888628 24684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:26.889734 24684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:26.891310 24684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:26.891443 24684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:26.891623 24684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:26.891722 24684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:26.928009 24684 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:38301
I20260807 18:26:26.928082 24861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:38301 every 8 connection(s)
I20260807 18:26:26.929358 24684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:26.935920 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 24684
I20260807 18:26:26.936137 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:26.936630 30323 external_mini_cluster.cc:1428] Setting key 8540fc2db39d740074b16e9d272c1bd2
I20260807 18:26:26.939443 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--builtin_ntp_servers=127.29.156.212:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:26.943156 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:26.932055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:45103 (local address 127.29.156.254:37713)
0807 18:26:26.932564 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:26.932571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:26.933777 (+ 1206us) server_negotiation.cc:407] Connection header received
0807 18:26:26.935584 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:26.935589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:26.935686 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:26.935820 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:26.937137 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.937988 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.939793 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.940572 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.940750 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:26.942166 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:26.942174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:26.942182 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:26.942206 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:26.942259 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:26.942277 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:26.942375 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:26.942586 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 18:26:26.942695 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":235,"threads_started":1}
I20260807 18:26:26.944006 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:26.932704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37713 (local address 127.29.156.193:45103)
0807 18:26:26.933614 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:26.933661 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:26.935281 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:26.936148 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:26.936162 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:26.936294 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:26.936950 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:26.936974 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.938166 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.938170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:26.939638 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:26.939656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:26.940815 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:26.940820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:26.941108 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:26.941843 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:26.941845 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:26.941890 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:26.941966 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:26.942380 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:26.942388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:26.942394 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:26:26.942560 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:26:26.942844 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":280,"threads_started":1}
I20260807 18:26:26.945686 24862 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37713
I20260807 18:26:26.946003 24862 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:26.946965 24862 heartbeater.cc:511] Master 127.29.156.254:37713 requested a full tablet report, sending...
I20260807 18:26:26.949244 24609 ts_manager.cc:194] Registered new tserver with Master: e951f25d25fb4e2aa3ae248a525edb38 (127.29.156.193:38301)
I20260807 18:26:26.950749 24609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.193:45103
W20260807 18:26:27.296639 24870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:27.296973 24870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:27.297152 24870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:27.304967 24870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:27.305109 24870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:27.305282 24870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:27.314507 24870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:27.315867 24870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:27.318360 24870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:27.336251 24881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:27.336325 24882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:27.336308 24884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:27.337251 24870 server_base.cc:1034] running on GCE node
I20260807 18:26:27.337870 24870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:27.338805 24870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:27.340055 24870 hybrid_clock.cc:648] HybridClock initialized: now 1786127187339940 us; error 60 us; skew 500 ppm
I20260807 18:26:27.340408 24870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:27.347090 24870 webserver.cc:460] Webserver started at http://127.29.156.194:43873/ using document root <none> and password file <none>
I20260807 18:26:27.347913 24870 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:27.348037 24870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:27.348387 24870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:27.351468 24870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "56a7af2c6e824338b56544230ce84f72"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "6ed9e35a958e421a47c52314d910ab8f"
server_key_iv: "850abb84ce5183e10dd61db9e28599ad"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.352442 24870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "56a7af2c6e824338b56544230ce84f72"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "6ed9e35a958e421a47c52314d910ab8f"
server_key_iv: "850abb84ce5183e10dd61db9e28599ad"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.357730 24870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:26:27.361459 24892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.363088 24870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:26:27.363260 24870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "56a7af2c6e824338b56544230ce84f72"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "6ed9e35a958e421a47c52314d910ab8f"
server_key_iv: "850abb84ce5183e10dd61db9e28599ad"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.363480 24870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:27.376629 24870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:27.377524 24870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:27.377856 24870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:27.378887 24870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:27.380529 24870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:27.380621 24870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.380784 24870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:27.380878 24870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.416193 24870 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:37941
I20260807 18:26:27.416288 25049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:37941 every 8 connection(s)
I20260807 18:26:27.417582 24870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:27.419620 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 24870
I20260807 18:26:27.419806 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:27.420339 30323 external_mini_cluster.cc:1428] Setting key 44f3c970bfa468306def093ef33a81a5
I20260807 18:26:27.423465 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--builtin_ntp_servers=127.29.156.212:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:27.432060 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.420948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:53019 (local address 127.29.156.254:37713)
0807 18:26:27.421161 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.421188 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.422884 (+ 1696us) server_negotiation.cc:407] Connection header received
0807 18:26:27.424907 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.424913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.425010 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.425133 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:27.426272 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.426996 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.428703 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.429246 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.429392 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.430975 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:27.430984 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:27.430991 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:27.431029 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:27.431101 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:27.431122 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:27.431221 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.431473 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.431623 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:26:27.433115 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.421569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37713 (local address 127.29.156.194:53019)
0807 18:26:27.422725 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:27.422771 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:27.424626 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:27.425391 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:27.425399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:27.425499 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:27.426093 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.426111 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.427174 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.427180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.428540 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.428554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.429502 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.429511 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.429748 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.430528 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:27.430531 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:27.430617 (+ 86us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:27.430734 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:27.431251 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:27.431261 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:27.431268 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:26:27.431426 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:26:27.431879 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":345,"threads_started":1}
I20260807 18:26:27.434863 25050 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37713
I20260807 18:26:27.435194 25050 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:27.436051 25050 heartbeater.cc:511] Master 127.29.156.254:37713 requested a full tablet report, sending...
I20260807 18:26:27.437615 24609 ts_manager.cc:194] Registered new tserver with Master: 56a7af2c6e824338b56544230ce84f72 (127.29.156.194:37941)
I20260807 18:26:27.438974 24609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.194:53019
W20260807 18:26:27.780190 25053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:27.780529 25053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:27.780699 25053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:27.788323 25053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:27.788504 25053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:27.788697 25053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:27.800391 25053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:27.801966 25053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:27.804093 25053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:27.819410 25059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:27.819391 25058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:27.819564 25063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:27.820638 25053 server_base.cc:1034] running on GCE node
I20260807 18:26:27.821507 25053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:27.822793 25053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:27.824081 25053 hybrid_clock.cc:648] HybridClock initialized: now 1786127187824026 us; error 55 us; skew 500 ppm
I20260807 18:26:27.824478 25053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:27.827486 25053 webserver.cc:460] Webserver started at http://127.29.156.195:32969/ using document root <none> and password file <none>
I20260807 18:26:27.828353 25053 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:27.828507 25053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:27.828938 25053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:27.832213 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "a0b7ab31fe6d4bee9bf0301450d25b68"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "9390661372e55e4edf3f0b4c131596ae"
server_key_iv: "16d2a50288766ba8cfe98a99f38ed7d7"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.832973 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "a0b7ab31fe6d4bee9bf0301450d25b68"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "9390661372e55e4edf3f0b4c131596ae"
server_key_iv: "16d2a50288766ba8cfe98a99f38ed7d7"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.838181 25053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:26:27.842034 25070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.843487 25053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:26:27.843719 25053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "a0b7ab31fe6d4bee9bf0301450d25b68"
format_stamp: "Formatted at 2026-08-07 18:26:27 on dist-test-slave-sb2z"
server_key: "9390661372e55e4edf3f0b4c131596ae"
server_key_iv: "16d2a50288766ba8cfe98a99f38ed7d7"
server_key_version: "encryptionkey@0"
I20260807 18:26:27.843914 25053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:27.868948 25053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:27.869781 25053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:27.870141 25053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:27.871152 25053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:27.872717 25053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:27.872805 25053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.872879 25053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:27.872936 25053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:27.908399 25053 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:33697
I20260807 18:26:27.908607 25222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:33697 every 8 connection(s)
I20260807 18:26:27.909765 25053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:27.911250 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 25053
I20260807 18:26:27.911460 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:27.912163 30323 external_mini_cluster.cc:1428] Setting key b9ba4c3958cf7464f5152166393fbc84
I20260807 18:26:27.923658 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.913302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:42259 (local address 127.29.156.254:37713)
0807 18:26:27.913526 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.913533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.914990 (+ 1457us) server_negotiation.cc:407] Connection header received
0807 18:26:27.916988 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.916994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.917088 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.917199 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:27.918256 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.919024 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.920750 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.921189 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.921332 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.922777 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:27.922782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:27.922786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:27.922804 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:27.922836 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:27.922849 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:27.922950 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.923181 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.923286 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:26:27.924695 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.913751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:37713 (local address 127.29.156.195:42259)
0807 18:26:27.914837 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:27.914884 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:27.916703 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:27.917437 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:27.917446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:27.917544 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:27.918088 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.918105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.919189 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.919196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.920598 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.920611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.921425 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.921432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.921700 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.922435 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:27.922438 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:27.922483 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:27.922568 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:27.922948 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:27.922959 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:27.922969 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:26:27.923171 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 18:26:27.923618 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":367,"threads_started":1}
I20260807 18:26:27.926470 25223 heartbeater.cc:347] Connected to a master server at 127.29.156.254:37713
I20260807 18:26:27.926856 25223 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:27.928081 25223 heartbeater.cc:511] Master 127.29.156.254:37713 requested a full tablet report, sending...
I20260807 18:26:27.929629 24609 ts_manager.cc:194] Registered new tserver with Master: a0b7ab31fe6d4bee9bf0301450d25b68 (127.29.156.195:33697)
I20260807 18:26:27.930562 24609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.195:42259
I20260807 18:26:27.940953 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:27.952689 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.945809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.29.156.254:37713)
0807 18:26:27.946122 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.946129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.946476 (+ 347us) server_negotiation.cc:407] Connection header received
0807 18:26:27.946670 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.946674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.946789 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.946900 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:27.947400 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.948279 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.949919 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.950440 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.950559 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.951850 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:27.951855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:27.951859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:27.951878 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:27.951911 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:27.951924 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:27.952054 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.952297 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.952362 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:26:27.954384 24862 heartbeater.cc:503] Master 127.29.156.254:37713 was elected leader, sending a full tablet report...
I20260807 18:26:27.957228 24609 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:27.965101 24609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:27.990625 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.980827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:37941 (local address 127.0.0.1:42000)
0807 18:26:27.981686 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:27.981710 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:27.981844 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:27.983403 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:27.983414 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:27.983449 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:27.983963 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.983978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.985773 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.985784 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.988145 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.988156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.989536 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.989543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.989762 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.989827 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:26:27.989982 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:26:27.990180 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":30,"thread_start_us":803,"threads_started":1}
I20260807 18:26:27.990592 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.981419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.29.156.194:37941)
0807 18:26:27.982772 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.982780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.982809 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:26:27.982917 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.982928 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.983160 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.984033 (+ 873us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:27.984269 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.985588 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.988388 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.989319 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.989534 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.989771 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.990052 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.990228 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1197,"thread_start_us":1052,"threads_started":1}
I20260807 18:26:27.991151 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.980199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38301 (local address 127.0.0.1:44886)
0807 18:26:27.980792 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:27.980813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:27.980960 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:27.983180 (+ 2220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:27.983185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:27.983217 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:27.983763 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.983779 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.985411 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.985417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.988118 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.988134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.990225 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.990232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.990446 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.990501 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 18:26:27.990593 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:26:27.990746 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":127,"thread_start_us":300,"threads_started":1}
I20260807 18:26:27.991691 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.978781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33697 (local address 127.0.0.1:53882)
0807 18:26:27.979847 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:27.979911 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:27.980127 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:27.981059 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:27.981067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:27.981100 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:27.981627 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.981645 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.985008 (+ 3363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.985014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.987766 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:27.987781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.989801 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.989811 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:27.990009 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.990074 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:26:27.991140 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0807 18:26:27.991297 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":174,"thread_start_us":472,"threads_started":1}
I20260807 18:26:27.992515 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.980746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.29.156.193:38301)
0807 18:26:27.982465 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.982477 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.982513 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:26:27.982619 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.982626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.982898 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.983175 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:27.983984 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.985255 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.988358 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.989982 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.991484 (+ 1502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.991817 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.991898 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.992118 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1519,"thread_start_us":1383,"threads_started":1}
I20260807 18:26:27.992550 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:27.979170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.29.156.195:33697)
0807 18:26:27.980318 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:27.980330 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:27.980380 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:26:27.980513 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:27.980520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:27.980784 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:27.982758 (+ 1974us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:27.983140 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.984835 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.987986 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:27.989536 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:27.991711 (+ 2175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:27.991922 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:27.991992 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:26:27.992153 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":780,"threads_started":1}
I20260807 18:26:27.995839 24953 tablet_service.cc:1467] Processing CreateTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb (DEFAULT_TABLE table=test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:27.995849 24774 tablet_service.cc:1467] Processing CreateTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb (DEFAULT_TABLE table=test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:27.996079 25130 tablet_service.cc:1467] Processing CreateTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb (DEFAULT_TABLE table=test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:27.997481 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d05943a24dc402b938cbbc918cfbcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:27.997500 24953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d05943a24dc402b938cbbc918cfbcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:27.997499 25130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d05943a24dc402b938cbbc918cfbcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:28.014133 25251 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Bootstrap starting.
I20260807 18:26:28.016961 25249 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Bootstrap starting.
I20260807 18:26:28.018867 25251 tablet_bootstrap.cc:654] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:28.019999 25252 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Bootstrap starting.
I20260807 18:26:28.021534 25249 tablet_bootstrap.cc:654] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:28.021690 25251 log.cc:824] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:28.022819 25249 log.cc:824] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:28.024350 25252 tablet_bootstrap.cc:654] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:28.025312 25251 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: No bootstrap required, opened a new log
I20260807 18:26:28.025488 25252 log.cc:824] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:28.025655 25251 ts_tablet_manager.cc:1397] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:26:28.026083 25249 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: No bootstrap required, opened a new log
I20260807 18:26:28.026418 25249 ts_tablet_manager.cc:1397] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 18:26:28.028537 25252 tablet_bootstrap.cc:492] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: No bootstrap required, opened a new log
I20260807 18:26:28.028829 25252 ts_tablet_manager.cc:1397] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:26:28.037261 25251 raft_consensus.cc:348] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.037261 25249 raft_consensus.cc:348] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.037774 25249 raft_consensus.cc:374] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:28.037774 25251 raft_consensus.cc:374] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:28.037889 25251 raft_consensus.cc:729] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a7af2c6e824338b56544230ce84f72, State: Initialized, Role: FOLLOWER
I20260807 18:26:28.037925 25249 raft_consensus.cc:729] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f25d25fb4e2aa3ae248a525edb38, State: Initialized, Role: FOLLOWER
I20260807 18:26:28.038029 25252 raft_consensus.cc:348] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.038573 25252 raft_consensus.cc:374] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:28.038709 25252 raft_consensus.cc:729] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b7ab31fe6d4bee9bf0301450d25b68, State: Initialized, Role: FOLLOWER
I20260807 18:26:28.038591 25249 consensus_queue.cc:260] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.038591 25251 consensus_queue.cc:260] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.039405 25252 consensus_queue.cc:260] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.040905 25249 ts_tablet_manager.cc:1428] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 18:26:28.042146 25223 heartbeater.cc:503] Master 127.29.156.254:37713 was elected leader, sending a full tablet report...
I20260807 18:26:28.042910 25252 ts_tablet_manager.cc:1428] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 18:26:28.044535 25050 heartbeater.cc:503] Master 127.29.156.254:37713 was elected leader, sending a full tablet report...
I20260807 18:26:28.046099 25251 ts_tablet_manager.cc:1428] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Time spent starting tablet: real 0.020s user 0.015s sys 0.005s
W20260807 18:26:28.162119 25224 tablet.cc:2369] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:28.169811 25051 tablet.cc:2369] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:28.181970 24863 tablet.cc:2369] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:28.260736 25258 raft_consensus.cc:482] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:28.261229 25258 raft_consensus.cc:504] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.263185 25258 leader_election.cc:288] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0b7ab31fe6d4bee9bf0301450d25b68 (127.29.156.195:33697), e951f25d25fb4e2aa3ae248a525edb38 (127.29.156.193:38301)
I20260807 18:26:28.270815 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.263799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:37015 (local address 127.29.156.195:33697)
0807 18:26:28.264048 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.264055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.264508 (+ 453us) server_negotiation.cc:407] Connection header received
0807 18:26:28.264923 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.264927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.264988 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.265100 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:28.265793 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.266837 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.269062 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.269846 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.270063 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.270142 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.270426 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.270567 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:26:28.270877 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.264645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:58579 (local address 127.29.156.193:38301)
0807 18:26:28.264839 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.264843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.265275 (+ 432us) server_negotiation.cc:407] Connection header received
0807 18:26:28.265460 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.265466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.265578 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.265683 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:28.266154 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.267127 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.269137 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.269929 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.270120 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.270187 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.270494 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.270628 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:26:28.270929 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.263635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:33697 (local address 127.29.156.194:37015)
0807 18:26:28.264380 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:28.264401 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:28.264617 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:28.265121 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:28.265125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:28.265152 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:28.265613 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:28.265630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.267013 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.267018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:28.268867 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:28.268880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.270059 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.270066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:28.270267 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.270294 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:26:28.270399 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:26:28.270544 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":9,"thread_start_us":379,"threads_started":1}
I20260807 18:26:28.271198 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.264420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:38301 (local address 127.29.156.194:58579)
0807 18:26:28.265187 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:28.265201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:28.265327 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:28.265701 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:28.265704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:28.265723 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:28.266003 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:28.266010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.267278 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.267281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:28.268876 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:28.268889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.270145 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.270151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:28.270322 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.270345 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:26:28.270450 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:26:28.270617 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":32,"thread_start_us":449,"threads_started":1}
I20260807 18:26:28.272038 25162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d05943a24dc402b938cbbc918cfbcbb" candidate_uuid: "56a7af2c6e824338b56544230ce84f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" is_pre_election: true
I20260807 18:26:28.272234 24798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d05943a24dc402b938cbbc918cfbcbb" candidate_uuid: "56a7af2c6e824338b56544230ce84f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f25d25fb4e2aa3ae248a525edb38" is_pre_election: true
I20260807 18:26:28.272594 25162 raft_consensus.cc:2454] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56a7af2c6e824338b56544230ce84f72 in term 0.
I20260807 18:26:28.272773 24798 raft_consensus.cc:2454] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56a7af2c6e824338b56544230ce84f72 in term 0.
I20260807 18:26:28.273300 24894 leader_election.cc:302] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56a7af2c6e824338b56544230ce84f72, a0b7ab31fe6d4bee9bf0301450d25b68; no voters:
I20260807 18:26:28.273699 25258 raft_consensus.cc:2781] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:28.273864 25258 raft_consensus.cc:482] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:28.273975 25258 raft_consensus.cc:3037] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:28.276310 25258 raft_consensus.cc:504] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.276887 25258 leader_election.cc:288] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [CANDIDATE]: Term 1 election: Requested vote from peers a0b7ab31fe6d4bee9bf0301450d25b68 (127.29.156.195:33697), e951f25d25fb4e2aa3ae248a525edb38 (127.29.156.193:38301)
I20260807 18:26:28.277309 25162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d05943a24dc402b938cbbc918cfbcbb" candidate_uuid: "56a7af2c6e824338b56544230ce84f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68"
I20260807 18:26:28.277393 24798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d05943a24dc402b938cbbc918cfbcbb" candidate_uuid: "56a7af2c6e824338b56544230ce84f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f25d25fb4e2aa3ae248a525edb38"
I20260807 18:26:28.277549 25162 raft_consensus.cc:3037] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:28.277594 24798 raft_consensus.cc:3037] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:28.280424 25162 raft_consensus.cc:2454] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56a7af2c6e824338b56544230ce84f72 in term 1.
I20260807 18:26:28.280498 24798 raft_consensus.cc:2454] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56a7af2c6e824338b56544230ce84f72 in term 1.
I20260807 18:26:28.280861 24894 leader_election.cc:302] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56a7af2c6e824338b56544230ce84f72, a0b7ab31fe6d4bee9bf0301450d25b68; no voters:
I20260807 18:26:28.281219 25258 raft_consensus.cc:2781] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:28.281708 25258 raft_consensus.cc:686] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 1 LEADER]: Becoming Leader. State: Replica: 56a7af2c6e824338b56544230ce84f72, State: Running, Role: LEADER
I20260807 18:26:28.282186 25258 consensus_queue.cc:237] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } }
I20260807 18:26:28.286978 24609 catalog_manager.cc:5495] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56a7af2c6e824338b56544230ce84f72 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "56a7af2c6e824338b56544230ce84f72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56a7af2c6e824338b56544230ce84f72" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37941 } health_report { overall_health: HEALTHY } } }
I20260807 18:26:28.363684 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.357968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.29.156.194:37941)
0807 18:26:28.358215 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.358220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.358240 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:26:28.358349 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.358353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.358441 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.358586 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:28.358920 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.359706 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.361243 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.361716 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.361837 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.362468 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:28.362968 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:28.363114 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.363263 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.363329 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260807 18:26:28.372503 24798 raft_consensus.cc:1261] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 1 FOLLOWER]: Refusing update from remote peer 56a7af2c6e824338b56544230ce84f72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:28.372604 25162 raft_consensus.cc:1261] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 1 FOLLOWER]: Refusing update from remote peer 56a7af2c6e824338b56544230ce84f72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:28.373643 25262 consensus_queue.cc:1035] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f25d25fb4e2aa3ae248a525edb38" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:28.374104 25258 consensus_queue.cc:1035] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0b7ab31fe6d4bee9bf0301450d25b68" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 33697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:28.391712 25268 mvcc.cc:205] Tried to move back new op lower bound from 7315976963563421696 to 7315976963208138752. Current Snapshot: MvccSnapshot[applied={T|T < 7315976963563421696}]
I20260807 18:26:28.400161 24607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:28.400463 24607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:28.404456 24607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c00cc27d2042a4b0b8d9279fa13cfc: Sending DeleteTablet for 3 replicas of tablet 7d05943a24dc402b938cbbc918cfbcbb
I20260807 18:26:28.405795 25130 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:28 UTC) from {username='slave'} at 127.0.0.1:53882
I20260807 18:26:28.405951 24953 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:28 UTC) from {username='slave'} at 127.0.0.1:42000
I20260807 18:26:28.406680 24774 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d05943a24dc402b938cbbc918cfbcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:28 UTC) from {username='slave'} at 127.0.0.1:44886
I20260807 18:26:28.409518 25277 tablet_replica.cc:330] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: stopping tablet replica
I20260807 18:26:28.410107 25277 raft_consensus.cc:2229] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:28.410246 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 24684
I20260807 18:26:28.410923 25277 raft_consensus.cc:2258] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:28.413786 25278 tablet_replica.cc:330] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: stopping tablet replica
I20260807 18:26:28.414361 25278 raft_consensus.cc:2229] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:28.414999 25278 raft_consensus.cc:2258] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:28.417672 25278 ts_tablet_manager.cc:1905] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:28.424176 25276 tablet_replica.cc:330] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: stopping tablet replica
I20260807 18:26:28.424644 25276 raft_consensus.cc:2229] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:28.425204 25276 raft_consensus.cc:2258] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:28.425880 25277 ts_tablet_manager.cc:1905] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:28.426988 25278 ts_tablet_manager.cc:1918] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:28.427196 25278 log.cc:1196] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/7d05943a24dc402b938cbbc918cfbcbb
I20260807 18:26:28.427817 25278 ts_tablet_manager.cc:1939] T 7d05943a24dc402b938cbbc918cfbcbb P e951f25d25fb4e2aa3ae248a525edb38: Deleting consensus metadata
I20260807 18:26:28.429876 24585 catalog_manager.cc:4841] TS e951f25d25fb4e2aa3ae248a525edb38 (127.29.156.193:38301): tablet 7d05943a24dc402b938cbbc918cfbcbb (table test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]) successfully deleted
I20260807 18:26:28.434585 25277 ts_tablet_manager.cc:1918] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:28.434896 25277 log.cc:1196] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/7d05943a24dc402b938cbbc918cfbcbb
I20260807 18:26:28.435451 25276 ts_tablet_manager.cc:1905] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:28.435706 25277 ts_tablet_manager.cc:1939] T 7d05943a24dc402b938cbbc918cfbcbb P 56a7af2c6e824338b56544230ce84f72: Deleting consensus metadata
I20260807 18:26:28.437604 24585 catalog_manager.cc:4841] TS 56a7af2c6e824338b56544230ce84f72 (127.29.156.194:37941): tablet 7d05943a24dc402b938cbbc918cfbcbb (table test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]) successfully deleted
I20260807 18:26:28.442013 25276 ts_tablet_manager.cc:1918] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:28.442193 25276 log.cc:1196] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/7d05943a24dc402b938cbbc918cfbcbb
I20260807 18:26:28.442696 25276 ts_tablet_manager.cc:1939] T 7d05943a24dc402b938cbbc918cfbcbb P a0b7ab31fe6d4bee9bf0301450d25b68: Deleting consensus metadata
I20260807 18:26:28.444481 24585 catalog_manager.cc:4841] TS a0b7ab31fe6d4bee9bf0301450d25b68 (127.29.156.195:33697): tablet 7d05943a24dc402b938cbbc918cfbcbb (table test-table [id=3df44e8f00bc4c94821c5cd3f4068f01]) successfully deleted
I20260807 18:26:28.450347 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.412379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.29.156.193:38301)
0807 18:26:28.412579 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.412586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.441631 (+ 29045us) server_negotiation.cc:407] Connection header received
0807 18:26:28.441958 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.441965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.442079 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.442159 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:28.442992 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.444194 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.446345 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.447039 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.447182 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.448683 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:28.448696 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:28.448714 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:28.449491 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:28.449552 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:28.449577 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:28.449686 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.449858 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.449989 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:26:28.451107 24850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:28.545310 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 24684
I20260807 18:26:28.573257 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 24870
I20260807 18:26:28.581609 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.574261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42022 (local address 127.29.156.194:37941)
0807 18:26:28.574445 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.574449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.574467 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:26:28.574641 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.574645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.574726 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.574817 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:28.575312 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.576214 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.578160 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.578635 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.578741 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.580106 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:28.580115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:28.580130 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:28.580789 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:28.580840 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:28.580859 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:28.580962 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.581146 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.581232 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:26:28.582481 25043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:28.701817 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 24870
I20260807 18:26:28.727643 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 25053
I20260807 18:26:28.735140 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:28.728500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.29.156.195:33697)
0807 18:26:28.728747 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:28.728753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:28.728773 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:26:28.728839 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:28.728843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:28.728992 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:28.729082 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:28.729519 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.730393 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.732044 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:28.732479 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:28.732585 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:28.733643 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:28.733659 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:28.733675 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:28.734286 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:28.734342 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:28.734365 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:28.734469 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:28.734605 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:26:28.734704 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:26:28.735908 25215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:28.827876 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 25053
I20260807 18:26:28.854074 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 24557
I20260807 18:26:28.855456 24644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:28.972182 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 24557
2026-08-07T18:26:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3357 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:29 dist-test-slave-sb2z krb5kdc[25289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:29 dist-test-slave-sb2z krb5kdc[25289](info): set up 2 sockets
Aug 07 18:26:29 dist-test-slave-sb2z krb5kdc[25289](info): commencing operation
krb5kdc: starting...
W20260807 18:26:30.325276 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:30 dist-test-slave-sb2z krb5kdc[25289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:26:30.683969 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:35965
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:35965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:31.041159 25304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:31.041473 25304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:31.041568 25304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:31.048971 25304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:31.049108 25304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:31.049162 25304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:31.049226 25304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:31.049273 25304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:31.059315 25304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:35965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:35965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:31.060741 25304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:31.062053 25304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:31.073359 25312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:31.073904 25313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:31.074965 25316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:31.075557 25304 server_base.cc:1034] running on GCE node
I20260807 18:26:31.076357 25304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:31.077834 25304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:31.079046 25304 hybrid_clock.cc:648] HybridClock initialized: now 1786127191079010 us; error 30 us; skew 500 ppm
I20260807 18:26:31.079386 25304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:31 dist-test-slave-sb2z krb5kdc[25289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127191, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:31.082564 25304 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:26:31.084141 25304 webserver.cc:460] Webserver started at http://127.29.156.254:42249/ using document root <none> and password file <none>
I20260807 18:26:31.084815 25304 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:31.084915 25304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:31.085187 25304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:31.088353 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "e900898792684491aa785c224e9ffd6d"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "0c39e05b42619c43c229f65969cafc22"
server_key_iv: "526bcf4eac9f2f51586f1937060599b8"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.089071 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "e900898792684491aa785c224e9ffd6d"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "0c39e05b42619c43c229f65969cafc22"
server_key_iv: "526bcf4eac9f2f51586f1937060599b8"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.094442 25304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:26:31.098064 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:31.099501 25304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:26:31.099720 25304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "e900898792684491aa785c224e9ffd6d"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "0c39e05b42619c43c229f65969cafc22"
server_key_iv: "526bcf4eac9f2f51586f1937060599b8"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.099892 25304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:31.115464 25304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:31.119758 25304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:31.119999 25304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:31.139832 25304 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:35965
I20260807 18:26:31.139850 25393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:35965 every 8 connection(s)
I20260807 18:26:31.141295 25304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:31.145334 25394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:31.149327 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 25304
I20260807 18:26:31.149518 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:31.150000 30323 external_mini_cluster.cc:1428] Setting key 2613ca71684bb669e803dc7343e0d608
I20260807 18:26:31.158061 25394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: Bootstrap starting.
I20260807 18:26:31.161579 25394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:31.162874 25394 log.cc:824] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:26:31 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:31.165997 25394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: No bootstrap required, opened a new log
I20260807 18:26:31.173082 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:31.151980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.29.156.254:35965)
0807 18:26:31.152802 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:31.152814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:31.152856 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:26:31.154545 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:31.154568 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:31.154979 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:31.155400 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:31.156911 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.158244 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.159611 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.160330 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.160553 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:31.165402 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:31.165410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:31.165425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:31.165461 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:31.168680 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:31.169405 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:31.169411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:31.169418 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:31.169607 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:31.170027 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:31.170031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:31.170033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:31.170625 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:31.170951 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:31.171453 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 18:26:31.171953 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":385,"threads_started":1}
I20260807 18:26:31.174299 25394 raft_consensus.cc:348] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } }
I20260807 18:26:31.174677 25394 raft_consensus.cc:374] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:31.174800 25394 raft_consensus.cc:729] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e900898792684491aa785c224e9ffd6d, State: Initialized, Role: FOLLOWER
I20260807 18:26:31.175455 25394 consensus_queue.cc:260] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } }
I20260807 18:26:31.175765 25394 raft_consensus.cc:388] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:31.175932 25394 raft_consensus.cc:482] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:31.176154 25394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:31.178395 25394 raft_consensus.cc:504] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } }
I20260807 18:26:31.178850 25394 leader_election.cc:302] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e900898792684491aa785c224e9ffd6d; no voters:
I20260807 18:26:31.179417 25394 leader_election.cc:288] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:31.179661 25400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:31.180462 25400 raft_consensus.cc:686] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [term 1 LEADER]: Becoming Leader. State: Replica: e900898792684491aa785c224e9ffd6d, State: Running, Role: LEADER
I20260807 18:26:31.181214 25400 consensus_queue.cc:237] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } }
I20260807 18:26:31.182000 25394 sys_catalog.cc:564] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:31.184516 25401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e900898792684491aa785c224e9ffd6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } } }
I20260807 18:26:31.184541 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e900898792684491aa785c224e9ffd6d. Latest consensus state: current_term: 1 leader_uuid: "e900898792684491aa785c224e9ffd6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e900898792684491aa785c224e9ffd6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 35965 } } }
I20260807 18:26:31.185007 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:31.185154 25401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:31.186796 25408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:31.193712 25408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:31.205436 25408 catalog_manager.cc:1347] Generated new cluster ID: 5e7e19084be9412781d83502c033e77a
I20260807 18:26:31.205586 25408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:31.219762 25408 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:31.221288 25408 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:31.233384 25408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: Generated new TSK 0
I20260807 18:26:31.234546 25408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.156.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
Entry for principal kudu/127.29.156.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab.
I20260807 18:26:31.324785 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:31.687773 25422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:31.688057 25422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:31.688253 25422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:31.695585 25422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:31.695739 25422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:31.695938 25422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:31.706156 25422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.193.keytab
--principal=kudu/127.29.156.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:31.707561 25422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:31.708985 25422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:31.722716 25428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:31.722716 25429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:31.722755 25434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:31.723758 25422 server_base.cc:1034] running on GCE node
I20260807 18:26:31.724388 25422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:31.725383 25422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:31.726649 25422 hybrid_clock.cc:648] HybridClock initialized: now 1786127191726589 us; error 57 us; skew 500 ppm
I20260807 18:26:31.726979 25422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:31 dist-test-slave-sb2z krb5kdc[25289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127191, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:31.730293 25422 init.cc:377] Logged in from keytab as kudu/127.29.156.193@KRBTEST.COM (short username kudu)
I20260807 18:26:31.731803 25422 webserver.cc:460] Webserver started at http://127.29.156.193:43085/ using document root <none> and password file <none>
I20260807 18:26:31.732501 25422 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:31.732606 25422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:31.732882 25422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:31.736289 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "b7e980039b284989a73f3bb6678fae41"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "d08a6dbdb14a3057235a869be76e9a11"
server_key_iv: "4f5642f32bf3cd988b0dec6556e821a2"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.737123 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "b7e980039b284989a73f3bb6678fae41"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "d08a6dbdb14a3057235a869be76e9a11"
server_key_iv: "4f5642f32bf3cd988b0dec6556e821a2"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.742537 25422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:26:31.746168 25445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:31.747715 25422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:26:31.747890 25422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "b7e980039b284989a73f3bb6678fae41"
format_stamp: "Formatted at 2026-08-07 18:26:31 on dist-test-slave-sb2z"
server_key: "d08a6dbdb14a3057235a869be76e9a11"
server_key_iv: "4f5642f32bf3cd988b0dec6556e821a2"
server_key_version: "encryptionkey@0"
I20260807 18:26:31.748119 25422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:31.761519 25422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:31.765750 25422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:31.766068 25422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:31.767089 25422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:31.768590 25422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:31.768666 25422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:31.768841 25422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:31.768904 25422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:31.805052 25422 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:42527
I20260807 18:26:31.805102 25596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:42527 every 8 connection(s)
I20260807 18:26:31.806418 25422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:31.811010 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 25422
I20260807 18:26:31.811233 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:31.811843 30323 external_mini_cluster.cc:1428] Setting key faa047979b601a7d0970acb1cd44b03b
Aug 07 18:26:31 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127191, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.193@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:31.832162 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:31.809448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:47125 (local address 127.29.156.254:35965)
0807 18:26:31.809937 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:31.809944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:31.811195 (+ 1251us) server_negotiation.cc:407] Connection header received
0807 18:26:31.813605 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:31.813611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:31.813694 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:31.813863 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:31.816061 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.816841 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.819377 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.819900 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.820017 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:31.824836 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:31.824845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:31.824852 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:31.824900 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:31.828567 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:31.829575 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:31.829584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:31.829590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:31.829767 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:31.830279 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:31.830285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:31.830288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:31.830576 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:31.830764 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:31.831492 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 18:26:31.831786 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":219,"threads_started":1}
I20260807 18:26:31.833335 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:31.810084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35965 (local address 127.29.156.193:47125)
0807 18:26:31.810965 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:31.811031 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:31.813304 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:31.814142 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:31.814159 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:31.814959 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:31.815862 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:31.815891 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.817024 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.817034 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:31.819213 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:31.819231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:31.820142 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:31.820153 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:31.820476 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:31.821296 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:31.821332 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:31.824569 (+ 3237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:31.828735 (+ 4166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:31.828747 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:31.828767 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:31.829419 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:31.829921 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:31.829925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:31.829928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:31.830127 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:31.830725 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:31.830733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:31.831086 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 18:26:31.831368 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:26:31.831959 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":280,"threads_started":1}
I20260807 18:26:31.835184 25597 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35965
I20260807 18:26:31.835601 25597 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:31.836411 25597 heartbeater.cc:511] Master 127.29.156.254:35965 requested a full tablet report, sending...
I20260807 18:26:31.839033 25352 ts_manager.cc:194] Registered new tserver with Master: b7e980039b284989a73f3bb6678fae41 (127.29.156.193:42527)
I20260807 18:26:31.841598 25352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.193@KRBTEST.COM'} at 127.29.156.193:47125
WARNING: no policy specified for kudu/127.29.156.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
Entry for principal kudu/127.29.156.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab.
I20260807 18:26:31.895177 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:32.252338 25603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:32.252621 25603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:32.252815 25603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:32.260250 25603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:32.260358 25603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:32.260529 25603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:32.270386 25603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.194.keytab
--principal=kudu/127.29.156.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:32.271935 25603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:32.273204 25603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:32.287082 25609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:32.287544 25613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:32.288360 25610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:32.288275 25603 server_base.cc:1034] running on GCE node
I20260807 18:26:32.289073 25603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:32.290302 25603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:32.291585 25603 hybrid_clock.cc:648] HybridClock initialized: now 1786127192291488 us; error 49 us; skew 500 ppm
I20260807 18:26:32.291918 25603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:32 dist-test-slave-sb2z krb5kdc[25289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127192, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:32.295400 25603 init.cc:377] Logged in from keytab as kudu/127.29.156.194@KRBTEST.COM (short username kudu)
I20260807 18:26:32.297042 25603 webserver.cc:460] Webserver started at http://127.29.156.194:36565/ using document root <none> and password file <none>
I20260807 18:26:32.297746 25603 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:32.297884 25603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:32.298230 25603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:32.301309 25603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "175cd3b3261a456d9cf6ef56b9fc7696"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "d44e7d803316d6db3292062dcd2dc8d5"
server_key_iv: "b98ad01b7ad2d81bc330ecf6e066348d"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.302083 25603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "175cd3b3261a456d9cf6ef56b9fc7696"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "d44e7d803316d6db3292062dcd2dc8d5"
server_key_iv: "b98ad01b7ad2d81bc330ecf6e066348d"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.307688 25603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:26:32.311239 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:32.312685 25603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:26:32.312870 25603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "175cd3b3261a456d9cf6ef56b9fc7696"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "d44e7d803316d6db3292062dcd2dc8d5"
server_key_iv: "b98ad01b7ad2d81bc330ecf6e066348d"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.313092 25603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:32.356226 25603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:32.361225 25603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:32.361603 25603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:32.362792 25603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:32.364385 25603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:32.364517 25603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:32.364696 25603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:32.364784 25603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:32.403455 25603 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:41017
I20260807 18:26:32.403505 25778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:41017 every 8 connection(s)
I20260807 18:26:32.405001 25603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:32.411960 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 25603
I20260807 18:26:32.412204 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:32.412736 30323 external_mini_cluster.cc:1428] Setting key fe6457aa193cfcf118b82c07e707e2ff
Aug 07 18:26:32 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127192, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.194@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:32.430038 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:32.407945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:50573 (local address 127.29.156.254:35965)
0807 18:26:32.408431 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:32.408436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:32.409587 (+ 1151us) server_negotiation.cc:407] Connection header received
0807 18:26:32.411784 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:32.411793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:32.411897 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:32.412030 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:32.413936 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:32.414921 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:32.416792 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:32.417740 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:32.417898 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:32.421664 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:32.421674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:32.421682 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:32.421741 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:32.425972 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:32.426941 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:32.426950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:32.426957 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:32.427135 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:32.427678 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:32.427687 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:32.427694 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:32.428113 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:32.428291 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:32.429172 (+ 881us) server_negotiation.cc:299] Negotiation successful
0807 18:26:32.429455 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260807 18:26:32.430915 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:32.408638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35965 (local address 127.29.156.194:50573)
0807 18:26:32.409384 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:32.409443 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:32.411392 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:32.412309 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:32.412325 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:32.412974 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:32.413730 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:32.413752 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:32.415120 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:32.415125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:32.416592 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:32.416605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:32.418041 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:32.418047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:32.418252 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:32.419011 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:32.419035 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:32.421353 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:32.426136 (+ 4783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:32.426147 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:32.426166 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:32.426778 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:32.427259 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:32.427266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:32.427272 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:32.427515 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:32.428257 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:32.428275 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:32.428705 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 18:26:32.428993 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 18:26:32.429531 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":303,"threads_started":1}
I20260807 18:26:32.432469 25782 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35965
I20260807 18:26:32.432799 25782 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:32.433547 25782 heartbeater.cc:511] Master 127.29.156.254:35965 requested a full tablet report, sending...
I20260807 18:26:32.435106 25352 ts_manager.cc:194] Registered new tserver with Master: 175cd3b3261a456d9cf6ef56b9fc7696 (127.29.156.194:41017)
I20260807 18:26:32.436621 25352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.194@KRBTEST.COM'} at 127.29.156.194:50573
WARNING: no policy specified for kudu/127.29.156.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
Entry for principal kudu/127.29.156.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab.
I20260807 18:26:32.494915 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:26:32.845045 25597 heartbeater.cc:503] Master 127.29.156.254:35965 was elected leader, sending a full tablet report...
W20260807 18:26:32.876899 25793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:32.877183 25793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:32.877337 25793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:32.885061 25793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:32.885171 25793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:32.885393 25793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:32.895567 25793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.195.keytab
--principal=kudu/127.29.156.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:35965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:32.896960 25793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:32.898371 25793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:32.913349 25799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:32.914033 25804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:32.914290 25800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:32.915331 25793 server_base.cc:1034] running on GCE node
I20260807 18:26:32.916086 25793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:32.917364 25793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:32.918601 25793 hybrid_clock.cc:648] HybridClock initialized: now 1786127192918523 us; error 70 us; skew 500 ppm
I20260807 18:26:32.919045 25793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:32 dist-test-slave-sb2z krb5kdc[25289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127192, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:32.923133 25793 init.cc:377] Logged in from keytab as kudu/127.29.156.195@KRBTEST.COM (short username kudu)
I20260807 18:26:32.925034 25793 webserver.cc:460] Webserver started at http://127.29.156.195:46129/ using document root <none> and password file <none>
I20260807 18:26:32.925849 25793 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:32.926026 25793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:32.926402 25793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:32.929857 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "f0efcab401b44a8b89949a7d9c21a02b"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "21f531b6ba96d6dc1a451a879751433f"
server_key_iv: "6b285f286673163dc29fb3d2bcef9a04"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.930944 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "f0efcab401b44a8b89949a7d9c21a02b"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "21f531b6ba96d6dc1a451a879751433f"
server_key_iv: "6b285f286673163dc29fb3d2bcef9a04"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.937289 25793 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:26:32.941751 25812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:32.943346 25793 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:26:32.943634 25793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "f0efcab401b44a8b89949a7d9c21a02b"
format_stamp: "Formatted at 2026-08-07 18:26:32 on dist-test-slave-sb2z"
server_key: "21f531b6ba96d6dc1a451a879751433f"
server_key_iv: "6b285f286673163dc29fb3d2bcef9a04"
server_key_version: "encryptionkey@0"
I20260807 18:26:32.943879 25793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:32.967829 25793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:32.972640 25793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:32.972956 25793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:32.974099 25793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:32.975618 25793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:32.975730 25793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:32.975888 25793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:32.975965 25793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:33.018944 25793 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:37451
I20260807 18:26:33.018990 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:37451 every 8 connection(s)
I20260807 18:26:33.020438 25793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:33.022526 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 25793
I20260807 18:26:33.022727 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:33.023226 30323 external_mini_cluster.cc:1428] Setting key 0bdf1b9c90bcfcf6306f30adbd7b6915
Aug 07 18:26:33 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127192, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.195@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:26:33.044703 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.023675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:60041 (local address 127.29.156.254:35965)
0807 18:26:33.024198 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.024202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.025590 (+ 1388us) server_negotiation.cc:407] Connection header received
0807 18:26:33.028072 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.028079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.028198 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.028321 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:33.029969 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.030794 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.032556 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.032958 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.033104 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.037473 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:33.037479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:33.037482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:33.037517 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:33.040824 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.041879 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.041884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.041889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.042101 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.042800 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.042803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.042805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.043077 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:33.043206 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.044233 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.044403 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":282,"threads_started":1}
I20260807 18:26:33.046540 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.024453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:35965 (local address 127.29.156.195:60041)
0807 18:26:33.025392 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.025450 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.027787 (+ 2337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.028584 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.028593 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.029226 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:33.029789 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.029811 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.030950 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.030956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.032382 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.032396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.033200 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.033208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.033451 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.034211 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:26:33.034234 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:33.037240 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:33.040995 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:33.041005 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:33.041029 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:33.041749 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:33.042248 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:26:33.042256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:26:33.042262 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:26:33.042655 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:26:33.043221 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:33.043232 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:33.043801 (+ 569us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.044142 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.044817 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":638,"threads_started":1}
I20260807 18:26:33.048123 25965 heartbeater.cc:347] Connected to a master server at 127.29.156.254:35965
I20260807 18:26:33.048456 25965 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:33.049266 25965 heartbeater.cc:511] Master 127.29.156.254:35965 requested a full tablet report, sending...
I20260807 18:26:33.050751 25352 ts_manager.cc:194] Registered new tserver with Master: f0efcab401b44a8b89949a7d9c21a02b (127.29.156.195:37451)
I20260807 18:26:33.051719 25352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.156.195@KRBTEST.COM'} at 127.29.156.195:60041
I20260807 18:26:33.054416 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:33.076941 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.060498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.29.156.254:35965)
0807 18:26:33.060732 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.060741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.061160 (+ 419us) server_negotiation.cc:407] Connection header received
0807 18:26:33.061310 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.061318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.061412 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.061539 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:33.063067 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.064822 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.066273 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.067123 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.067319 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.069859 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:33.069868 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:33.069875 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:33.069929 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:33.073786 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.074592 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.074597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.074600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.074745 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.075279 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.075284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.075286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.075785 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:33.076075 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.076404 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.076639 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:26:33.081573 25352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:33.089347 25352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:33.130445 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.102774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:42527 (local address 127.0.0.1:34682)
0807 18:26:33.104655 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.104721 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.104953 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.122290 (+ 17337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.122299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.122334 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:33.122856 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.122872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125349 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.125356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.127597 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.127613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.129236 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.129243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.129463 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.129542 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.129715 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.129926 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1478,"mutex_wait_us":35,"thread_start_us":1686,"threads_started":1}
I20260807 18:26:33.130901 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.103668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:37451 (local address 127.0.0.1:57992)
0807 18:26:33.122026 (+ 18358us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.122054 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.122309 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.123106 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.123111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.123150 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:33.123718 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.123733 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125913 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.125918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.127865 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.127881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.129751 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.129758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.129930 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.129964 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.130068 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.130208 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18195,"thread_start_us":18425,"threads_started":1}
I20260807 18:26:33.131035 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.104096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.29.156.195:37451)
0807 18:26:33.118796 (+ 14700us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.118811 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.122181 (+ 3370us) server_negotiation.cc:407] Connection header received
0807 18:26:33.122550 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.122559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.122861 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.123102 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:33.123963 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125718 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.128043 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.129526 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.129721 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.129887 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.130161 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.130338 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14406,"thread_start_us":14715,"threads_started":1}
I20260807 18:26:33.131343 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.102979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.29.156.193:42527)
0807 18:26:33.121303 (+ 18324us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.121318 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.121365 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:26:33.121522 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.121536 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.121927 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.123104 (+ 1177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:33.123379 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125159 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.127777 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.129008 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.129233 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.129435 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.130777 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.130965 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18019,"thread_start_us":17761,"threads_started":1}
I20260807 18:26:33.133244 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.105489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:41017 (local address 127.0.0.1:51792)
0807 18:26:33.122379 (+ 16890us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.122398 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.122539 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.123481 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.123487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.123541 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:33.123997 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.124006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125424 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.125432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.129338 (+ 3906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.129354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.132417 (+ 3063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.132422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.132621 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.132695 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.132776 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.132906 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16761,"thread_start_us":16201,"threads_started":1}
I20260807 18:26:33.133977 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.106138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.29.156.194:41017)
0807 18:26:33.121486 (+ 15348us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.121498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.122598 (+ 1100us) server_negotiation.cc:407] Connection header received
0807 18:26:33.122733 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.122739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.122911 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.123184 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:33.124166 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.125198 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.129540 (+ 4342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.131058 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.131237 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.131396 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.133489 (+ 2093us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.133691 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15164,"thread_start_us":15056,"threads_started":1}
I20260807 18:26:33.135608 25506 tablet_service.cc:1467] Processing CreateTablet for tablet 01f52ded69114774a014440f95ad3e99 (DEFAULT_TABLE table=test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:33.135607 25883 tablet_service.cc:1467] Processing CreateTablet for tablet 01f52ded69114774a014440f95ad3e99 (DEFAULT_TABLE table=test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:33.137243 25506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f52ded69114774a014440f95ad3e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:33.137260 25883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f52ded69114774a014440f95ad3e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:33.136889 25695 tablet_service.cc:1467] Processing CreateTablet for tablet 01f52ded69114774a014440f95ad3e99 (DEFAULT_TABLE table=test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:33.138056 25695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f52ded69114774a014440f95ad3e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:33.162724 25997 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Bootstrap starting.
I20260807 18:26:33.163749 25995 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Bootstrap starting.
I20260807 18:26:33.164691 25996 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Bootstrap starting.
I20260807 18:26:33.168617 25997 tablet_bootstrap.cc:654] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:33.169613 25996 tablet_bootstrap.cc:654] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:33.170086 25997 log.cc:824] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:33.170176 25995 tablet_bootstrap.cc:654] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:33.170825 25996 log.cc:824] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:33.171344 25995 log.cc:824] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:33.173879 25997 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: No bootstrap required, opened a new log
I20260807 18:26:33.174227 25997 ts_tablet_manager.cc:1397] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:26:33.174568 25995 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: No bootstrap required, opened a new log
I20260807 18:26:33.174683 25996 tablet_bootstrap.cc:492] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: No bootstrap required, opened a new log
I20260807 18:26:33.174872 25995 ts_tablet_manager.cc:1397] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 18:26:33.174993 25996 ts_tablet_manager.cc:1397] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260807 18:26:33.183099 25996 raft_consensus.cc:348] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.183637 25996 raft_consensus.cc:374] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:33.183771 25996 raft_consensus.cc:729] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e980039b284989a73f3bb6678fae41, State: Initialized, Role: FOLLOWER
I20260807 18:26:33.184420 25996 consensus_queue.cc:260] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.185776 25995 raft_consensus.cc:348] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.186219 25995 raft_consensus.cc:374] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:33.186290 25996 ts_tablet_manager.cc:1428] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260807 18:26:33.186329 25995 raft_consensus.cc:729] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0efcab401b44a8b89949a7d9c21a02b, State: Initialized, Role: FOLLOWER
I20260807 18:26:33.186954 25995 consensus_queue.cc:260] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.187199 25997 raft_consensus.cc:348] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.187757 25997 raft_consensus.cc:374] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:33.187860 25997 raft_consensus.cc:729] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175cd3b3261a456d9cf6ef56b9fc7696, State: Initialized, Role: FOLLOWER
I20260807 18:26:33.188179 25965 heartbeater.cc:503] Master 127.29.156.254:35965 was elected leader, sending a full tablet report...
I20260807 18:26:33.188476 25997 consensus_queue.cc:260] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.189689 25782 heartbeater.cc:503] Master 127.29.156.254:35965 was elected leader, sending a full tablet report...
I20260807 18:26:33.190455 25997 ts_tablet_manager.cc:1428] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260807 18:26:33.192130 25995 ts_tablet_manager.cc:1428] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260807 18:26:33.272651 25966 tablet.cc:2369] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:33.309182 25598 tablet.cc:2369] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:33.407366 25783 tablet.cc:2369] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:33.512329 26007 raft_consensus.cc:482] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:33.512687 26007 raft_consensus.cc:504] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.514530 26007 leader_election.cc:288] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7e980039b284989a73f3bb6678fae41 (127.29.156.193:42527), f0efcab401b44a8b89949a7d9c21a02b (127.29.156.195:37451)
I20260807 18:26:33.521528 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.514899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:51027 (local address 127.29.156.193:42527)
0807 18:26:33.515173 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.515181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.515985 (+ 804us) server_negotiation.cc:407] Connection header received
0807 18:26:33.516224 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.516228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.516338 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.516435 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:33.517089 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.517892 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.519723 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.520557 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.520749 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.520824 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.521141 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.521268 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:26:33.521799 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.514891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:42527 (local address 127.29.156.194:51027)
0807 18:26:33.515859 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.515882 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.516073 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.516470 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.516475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.516512 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:33.516925 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.516950 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.518022 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.518028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.519499 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.519511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.520767 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.520774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.520965 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.520989 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.521113 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.521251 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"spinlock_wait_cycles":2816,"thread_start_us":508,"threads_started":1}
I20260807 18:26:33.521842 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.515046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:45583 (local address 127.29.156.195:37451)
0807 18:26:33.515303 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.515311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.515995 (+ 684us) server_negotiation.cc:407] Connection header received
0807 18:26:33.516264 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.516268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.516329 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.516430 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:33.517227 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.518014 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.519845 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.520851 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.521051 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.521134 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.521450 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.521578 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 18:26:33.521996 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.514898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:37451 (local address 127.29.156.194:45583)
0807 18:26:33.515878 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:33.515894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:33.516080 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:33.516470 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:33.516475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:33.516549 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:33.517063 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.517072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.518181 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.518190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.519690 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:33.519703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.521104 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.521112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:33.521275 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.521297 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:26:33.521424 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:26:33.521556 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":9,"spinlock_wait_cycles":21760,"thread_start_us":471,"threads_started":1}
I20260807 18:26:33.522938 25910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f52ded69114774a014440f95ad3e99" candidate_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0efcab401b44a8b89949a7d9c21a02b" is_pre_election: true
I20260807 18:26:33.522958 25533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f52ded69114774a014440f95ad3e99" candidate_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e980039b284989a73f3bb6678fae41" is_pre_election: true
I20260807 18:26:33.523437 25910 raft_consensus.cc:2454] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175cd3b3261a456d9cf6ef56b9fc7696 in term 0.
I20260807 18:26:33.523466 25533 raft_consensus.cc:2454] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175cd3b3261a456d9cf6ef56b9fc7696 in term 0.
I20260807 18:26:33.524178 25627 leader_election.cc:302] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175cd3b3261a456d9cf6ef56b9fc7696, f0efcab401b44a8b89949a7d9c21a02b; no voters:
I20260807 18:26:33.524611 26007 raft_consensus.cc:2781] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:33.524784 26007 raft_consensus.cc:482] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:33.524914 26007 raft_consensus.cc:3037] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:33.527251 26007 raft_consensus.cc:504] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.528048 26007 leader_election.cc:288] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [CANDIDATE]: Term 1 election: Requested vote from peers b7e980039b284989a73f3bb6678fae41 (127.29.156.193:42527), f0efcab401b44a8b89949a7d9c21a02b (127.29.156.195:37451)
I20260807 18:26:33.528770 25910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f52ded69114774a014440f95ad3e99" candidate_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0efcab401b44a8b89949a7d9c21a02b"
I20260807 18:26:33.528762 25533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f52ded69114774a014440f95ad3e99" candidate_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e980039b284989a73f3bb6678fae41"
I20260807 18:26:33.529021 25910 raft_consensus.cc:3037] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:33.529019 25533 raft_consensus.cc:3037] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:33.532730 25910 raft_consensus.cc:2454] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175cd3b3261a456d9cf6ef56b9fc7696 in term 1.
I20260807 18:26:33.532748 25533 raft_consensus.cc:2454] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175cd3b3261a456d9cf6ef56b9fc7696 in term 1.
I20260807 18:26:33.533215 25627 leader_election.cc:302] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175cd3b3261a456d9cf6ef56b9fc7696, f0efcab401b44a8b89949a7d9c21a02b; no voters:
I20260807 18:26:33.533550 26007 raft_consensus.cc:2781] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:33.534081 26007 raft_consensus.cc:686] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 1 LEADER]: Becoming Leader. State: Replica: 175cd3b3261a456d9cf6ef56b9fc7696, State: Running, Role: LEADER
I20260807 18:26:33.534581 26007 consensus_queue.cc:237] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } }
I20260807 18:26:33.539734 25352 catalog_manager.cc:5495] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 reported cstate change: term changed from 0 to 1, leader changed from <none> to 175cd3b3261a456d9cf6ef56b9fc7696 (127.29.156.194). New cstate: current_term: 1 leader_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175cd3b3261a456d9cf6ef56b9fc7696" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 41017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 } health_report { overall_health: UNKNOWN } } }
I20260807 18:26:33.572360 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.566260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51794 (local address 127.29.156.194:41017)
0807 18:26:33.566488 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.566493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.566572 (+ 79us) server_negotiation.cc:407] Connection header received
0807 18:26:33.566791 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.566795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.566888 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.567052 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:33.567477 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.568412 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.569901 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.570428 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.570541 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.571092 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:33.571783 (+ 691us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:33.571921 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.572026 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.572106 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:26:33.581292 25533 raft_consensus.cc:1261] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 1 FOLLOWER]: Refusing update from remote peer 175cd3b3261a456d9cf6ef56b9fc7696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:33.581307 25910 raft_consensus.cc:1261] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 1 FOLLOWER]: Refusing update from remote peer 175cd3b3261a456d9cf6ef56b9fc7696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:33.582391 26010 consensus_queue.cc:1035] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e980039b284989a73f3bb6678fae41" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 42527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:33.582791 26007 consensus_queue.cc:1035] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0efcab401b44a8b89949a7d9c21a02b" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 37451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:33.595546 26015 mvcc.cc:205] Tried to move back new op lower bound from 7315976984898002944 to 7315976984722264064. Current Snapshot: MvccSnapshot[applied={T|T < 7315976984898002944}]
I20260807 18:26:33.611310 25352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:33.611655 25352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:33.615732 25352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e900898792684491aa785c224e9ffd6d: Sending DeleteTablet for 3 replicas of tablet 01f52ded69114774a014440f95ad3e99
I20260807 18:26:33.617069 25506 tablet_service.cc:1514] Processing DeleteTablet for tablet 01f52ded69114774a014440f95ad3e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:34682
I20260807 18:26:33.617163 25695 tablet_service.cc:1514] Processing DeleteTablet for tablet 01f52ded69114774a014440f95ad3e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:51792
I20260807 18:26:33.617308 25883 tablet_service.cc:1514] Processing DeleteTablet for tablet 01f52ded69114774a014440f95ad3e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:33 UTC) from {username='kudu', principal='kudu/127.29.156.254@KRBTEST.COM'} at 127.0.0.1:57992
I20260807 18:26:33.618697 26027 tablet_replica.cc:330] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: stopping tablet replica
I20260807 18:26:33.619320 26027 raft_consensus.cc:2229] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:33.620026 26027 raft_consensus.cc:2258] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:33.620461 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 25422
I20260807 18:26:33.621387 26026 tablet_replica.cc:330] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: stopping tablet replica
I20260807 18:26:33.621934 26026 raft_consensus.cc:2229] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:33.622859 26026 raft_consensus.cc:2258] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:33.623715 26025 tablet_replica.cc:330] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: stopping tablet replica
I20260807 18:26:33.623922 26027 ts_tablet_manager.cc:1905] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:33.624207 26025 raft_consensus.cc:2229] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:33.624780 26025 raft_consensus.cc:2258] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:33.625751 26026 ts_tablet_manager.cc:1905] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:33.630311 26025 ts_tablet_manager.cc:1905] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:33.634490 26026 ts_tablet_manager.cc:1918] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:33.634718 26026 log.cc:1196] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/01f52ded69114774a014440f95ad3e99
I20260807 18:26:33.635385 26026 ts_tablet_manager.cc:1939] T 01f52ded69114774a014440f95ad3e99 P 175cd3b3261a456d9cf6ef56b9fc7696: Deleting consensus metadata
I20260807 18:26:33.637570 25330 catalog_manager.cc:4841] TS 175cd3b3261a456d9cf6ef56b9fc7696 (127.29.156.194:41017): tablet 01f52ded69114774a014440f95ad3e99 (table test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]) successfully deleted
I20260807 18:26:33.640300 26027 ts_tablet_manager.cc:1918] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:33.640524 26027 log.cc:1196] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/01f52ded69114774a014440f95ad3e99
I20260807 18:26:33.641234 26027 ts_tablet_manager.cc:1939] T 01f52ded69114774a014440f95ad3e99 P f0efcab401b44a8b89949a7d9c21a02b: Deleting consensus metadata
I20260807 18:26:33.642915 26025 ts_tablet_manager.cc:1918] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:33.643194 26025 log.cc:1196] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126848590636-30323-0/minicluster-data/ts-0/wal/wals/01f52ded69114774a014440f95ad3e99
I20260807 18:26:33.643564 25333 catalog_manager.cc:4841] TS f0efcab401b44a8b89949a7d9c21a02b (127.29.156.195:37451): tablet 01f52ded69114774a014440f95ad3e99 (table test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]) successfully deleted
I20260807 18:26:33.643989 26025 ts_tablet_manager.cc:1939] T 01f52ded69114774a014440f95ad3e99 P b7e980039b284989a73f3bb6678fae41: Deleting consensus metadata
Aug 07 18:26:33 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.193@KRBTEST.COM
I20260807 18:26:33.645591 25330 catalog_manager.cc:4841] TS b7e980039b284989a73f3bb6678fae41 (127.29.156.193:42527): tablet 01f52ded69114774a014440f95ad3e99 (table test-table [id=9dc03aa5ea804d4c8c3f3434194729d6]) successfully deleted
I20260807 18:26:33.652456 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.623193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.29.156.193:42527)
0807 18:26:33.623568 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.623578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.631936 (+ 8358us) server_negotiation.cc:407] Connection header received
0807 18:26:33.632436 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.632443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.632523 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.632605 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:33.634696 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.635851 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.638186 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.638950 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.639108 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.645576 (+ 6468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:33.645591 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:33.645612 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:33.645659 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:33.648997 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.649943 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.649952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.649956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.650103 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.650589 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.650594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.650596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.650981 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:33.651228 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.651811 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.652041 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260807 18:26:33.653168 25590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:33.748163 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 25422
I20260807 18:26:33.775403 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 25603
Aug 07 18:26:33 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.194@KRBTEST.COM
I20260807 18:26:33.794559 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.776424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.29.156.194:41017)
0807 18:26:33.776652 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.776659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.776682 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:26:33.776752 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.776759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.776844 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.776961 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:33.778757 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.780022 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.781331 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.782112 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.782241 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.786695 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:33.786717 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:33.786739 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:33.786798 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:33.791211 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.792166 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.792176 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.792180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.792352 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.792733 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.792739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.792743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.793224 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:33.793582 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.793793 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.794035 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:26:33.795574 25772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:33.917495 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 25603
I20260807 18:26:33.944113 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 25793
Aug 07 18:26:33 dist-test-slave-sb2z krb5kdc[25289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.195@KRBTEST.COM
I20260807 18:26:33.961050 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:33.944918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58008 (local address 127.29.156.195:37451)
0807 18:26:33.945090 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:33.945095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:33.945186 (+ 91us) server_negotiation.cc:407] Connection header received
0807 18:26:33.945340 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:33.945344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:33.945421 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:33.945509 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:33.946920 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.947711 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.949257 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:33.949646 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:33.949739 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:33.954728 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:33.954743 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:33.954760 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:33.954799 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:33.958025 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.958792 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.958800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.958802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.958942 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:33.959311 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:33.959315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:33.959317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:33.959786 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:33.960026 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:33.960407 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:26:33.960590 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:26:33.961818 25960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:34.054870 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 25793
I20260807 18:26:34.080265 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 25304
I20260807 18:26:34.081640 25389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:34.196923 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 25304
2026-08-07T18:26:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5225 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:26:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:34Z Disabled control of system clock
I20260807 18:26:34.302207 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43563
--webserver_interface=127.29.156.254
--webserver_port=0
--builtin_ntp_servers=127.29.156.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:34.657935 26042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:34.658247 26042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:34.658392 26042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:34.667388 26042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:34.667516 26042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:34.667629 26042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:34.667716 26042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:34.667783 26042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:34.677603 26042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:34.679049 26042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:34.681099 26042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:34.694263 26052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:34.694314 26049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:34.694283 26050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:34.695823 26042 server_base.cc:1034] running on GCE node
I20260807 18:26:34.697062 26042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:34.698539 26042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:34.699751 26042 hybrid_clock.cc:648] HybridClock initialized: now 1786127194699680 us; error 71 us; skew 500 ppm
I20260807 18:26:34.700196 26042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:34.702570 26042 webserver.cc:460] Webserver started at http://127.29.156.254:39181/ using document root <none> and password file <none>
I20260807 18:26:34.703302 26042 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:34.703402 26042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:34.703778 26042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:34.706869 26042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "ccf078a5424844468464dde574bc0d86"
format_stamp: "Formatted at 2026-08-07 18:26:34 on dist-test-slave-sb2z"
server_key: "1b442cb30ef72ca6b8d949bf9249c5af"
server_key_iv: "1b014c2738eeb2c1ecb4321fd8ca5d34"
server_key_version: "encryptionkey@0"
I20260807 18:26:34.707897 26042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "ccf078a5424844468464dde574bc0d86"
format_stamp: "Formatted at 2026-08-07 18:26:34 on dist-test-slave-sb2z"
server_key: "1b442cb30ef72ca6b8d949bf9249c5af"
server_key_iv: "1b014c2738eeb2c1ecb4321fd8ca5d34"
server_key_version: "encryptionkey@0"
I20260807 18:26:34.713348 26042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:26:34.717406 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:34.718931 26042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:26:34.719184 26042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "ccf078a5424844468464dde574bc0d86"
format_stamp: "Formatted at 2026-08-07 18:26:34 on dist-test-slave-sb2z"
server_key: "1b442cb30ef72ca6b8d949bf9249c5af"
server_key_iv: "1b014c2738eeb2c1ecb4321fd8ca5d34"
server_key_version: "encryptionkey@0"
I20260807 18:26:34.719425 26042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:34.748809 26042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:34.749783 26042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:34.750118 26042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:34.770401 26042 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43563
I20260807 18:26:34.770421 26133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43563 every 8 connection(s)
I20260807 18:26:34.771862 26042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:34.775985 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:34.776118 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26042
I20260807 18:26:34.776341 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:34.776834 30323 external_mini_cluster.cc:1428] Setting key 316e069924dd068c92f36395b863ef85
I20260807 18:26:34.791689 26134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: Bootstrap starting.
I20260807 18:26:34.794569 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:34.778913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.29.156.254:43563)
0807 18:26:34.780248 (+ 1335us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:34.780263 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:34.780328 (+ 65us) server_negotiation.cc:407] Connection header received
0807 18:26:34.783237 (+ 2909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:34.783268 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:34.784005 (+ 737us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:34.784495 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:34.785106 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:34.787029 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:34.788739 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:34.789839 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:34.790018 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:34.791432 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:34.791444 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:34.791463 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:34.792271 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:34.792382 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:34.792409 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:34.792535 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:34.792802 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:26:34.793154 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":402,"threads_started":1}
I20260807 18:26:34.795915 26134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:34.797319 26134 log.cc:824] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:34.800593 26134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: No bootstrap required, opened a new log
I20260807 18:26:34.808250 26134 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } }
I20260807 18:26:34.808624 26134 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:34.808702 26134 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf078a5424844468464dde574bc0d86, State: Initialized, Role: FOLLOWER
I20260807 18:26:34.809286 26134 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } }
I20260807 18:26:34.809499 26134 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:34.809620 26134 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:34.809752 26134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:34.812057 26134 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } }
I20260807 18:26:34.812530 26134 leader_election.cc:302] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccf078a5424844468464dde574bc0d86; no voters:
I20260807 18:26:34.813414 26134 leader_election.cc:288] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:34.813558 26139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:34.814378 26139 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [term 1 LEADER]: Becoming Leader. State: Replica: ccf078a5424844468464dde574bc0d86, State: Running, Role: LEADER
I20260807 18:26:34.814908 26139 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } }
I20260807 18:26:34.815773 26134 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:34.818197 26141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccf078a5424844468464dde574bc0d86. Latest consensus state: current_term: 1 leader_uuid: "ccf078a5424844468464dde574bc0d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } } }
I20260807 18:26:34.818370 26140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccf078a5424844468464dde574bc0d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf078a5424844468464dde574bc0d86" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43563 } } }
I20260807 18:26:34.819324 26141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:34.819316 26140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:34.822077 26147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:34.827812 26147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:34.837446 26147 catalog_manager.cc:1347] Generated new cluster ID: 51b43b1858304db8a873484503af001b
I20260807 18:26:34.837585 26147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:34.860013 26147 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:34.861464 26147 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:34.874102 26147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: Generated new TSK 0
I20260807 18:26:34.875077 26147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:34.880009 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.193:0
--local_ip_for_outbound_sockets=127.29.156.193
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--builtin_ntp_servers=127.29.156.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:26:35.248298 26164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:35.248585 26164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:35.248677 26164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:35.256196 26164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:35.256309 26164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:35.256417 26164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.193
I20260807 18:26:35.265749 26164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.156.193
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.193
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:35.267050 26164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:35.268886 26164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:35.282612 26174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:35.282501 26170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:35.282656 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:35.284276 26164 server_base.cc:1034] running on GCE node
I20260807 18:26:35.284996 26164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:35.286142 26164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:35.287437 26164 hybrid_clock.cc:648] HybridClock initialized: now 1786127195287370 us; error 55 us; skew 500 ppm
I20260807 18:26:35.287834 26164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:35.290565 26164 webserver.cc:460] Webserver started at http://127.29.156.193:34087/ using document root <none> and password file <none>
I20260807 18:26:35.291311 26164 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:35.291405 26164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:35.291795 26164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:35.294806 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data/instance:
uuid: "9abf4d7e234c451391b85144259357c5"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "8156c474ee4bf35ac448c6f9c5f5ae0c"
server_key_iv: "1a2b7545890fca96215a134613e58ee4"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.295833 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance:
uuid: "9abf4d7e234c451391b85144259357c5"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "8156c474ee4bf35ac448c6f9c5f5ae0c"
server_key_iv: "1a2b7545890fca96215a134613e58ee4"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.301329 26164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:26:35.305068 26181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.306483 26164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:35.306684 26164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal
uuid: "9abf4d7e234c451391b85144259357c5"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "8156c474ee4bf35ac448c6f9c5f5ae0c"
server_key_iv: "1a2b7545890fca96215a134613e58ee4"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.306886 26164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:35.328593 26164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:35.329516 26164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:35.329844 26164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:35.330991 26164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:35.332549 26164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:35.332640 26164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.332793 26164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:35.332888 26164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.368422 26164 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.193:34101
I20260807 18:26:35.368480 26340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.193:34101 every 8 connection(s)
I20260807 18:26:35.369776 26164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/data/info.pb
I20260807 18:26:35.371963 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26164
I20260807 18:26:35.372146 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-0/wal/instance
I20260807 18:26:35.372678 30323 external_mini_cluster.cc:1428] Setting key ab7cee5ec461d970ee62ecd3efdf8426
I20260807 18:26:35.376467 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.194:0
--local_ip_for_outbound_sockets=127.29.156.194
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--builtin_ntp_servers=127.29.156.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:35.385474 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:35.372713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.193:48653 (local address 127.29.156.254:43563)
0807 18:26:35.373729 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:35.373736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:35.374816 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 18:26:35.377430 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:35.377436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:35.377501 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:35.377599 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:35.379089 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.379876 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.382379 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.382804 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.382963 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:35.384609 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:35.384614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:35.384617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:35.384636 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:35.384676 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:35.384688 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:35.384757 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:35.385076 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:26:35.385163 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":769,"threads_started":1}
I20260807 18:26:35.386880 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:35.373313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43563 (local address 127.29.156.193:48653)
0807 18:26:35.374615 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:35.374665 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:35.377110 (+ 2445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:35.377880 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:35.377895 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:35.378034 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:35.378886 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:35.378913 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.380056 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.380064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:35.382211 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:35.382229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.383053 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.383063 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:35.383394 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:35.384217 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:35.384222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:35.384285 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:35.384393 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:35.384784 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:35.384798 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:35.384810 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:26:35.385016 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:26:35.385446 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":449,"threads_started":1}
I20260807 18:26:35.388523 26341 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43563
I20260807 18:26:35.388850 26341 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:35.389649 26341 heartbeater.cc:511] Master 127.29.156.254:43563 requested a full tablet report, sending...
I20260807 18:26:35.391886 26086 ts_manager.cc:194] Registered new tserver with Master: 9abf4d7e234c451391b85144259357c5 (127.29.156.193:34101)
I20260807 18:26:35.394083 26086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.193:48653
W20260807 18:26:35.764957 26349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:35.765266 26349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:35.765354 26349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:35.774725 26349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:35.774852 26349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:35.775023 26349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.194
I20260807 18:26:35.790072 26349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.156.194
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.194
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:35.791752 26349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:35.793668 26349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:35.807883 26356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:35.807823 26357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:35.807844 26359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:35.809262 26349 server_base.cc:1034] running on GCE node
I20260807 18:26:35.809916 26349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:35.810925 26349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:35.812244 26349 hybrid_clock.cc:648] HybridClock initialized: now 1786127195812164 us; error 72 us; skew 500 ppm
I20260807 18:26:35.812616 26349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:35.815033 26349 webserver.cc:460] Webserver started at http://127.29.156.194:45187/ using document root <none> and password file <none>
I20260807 18:26:35.815930 26349 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:35.816083 26349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:35.816494 26349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:35.819618 26349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data/instance:
uuid: "b338b9407bea48e8a774579a77a56368"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "c0cbbdfbd3ee24be6900bc6257e21de7"
server_key_iv: "8c3e49b1fe1d5a1a0647e76949ea391d"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.820489 26349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance:
uuid: "b338b9407bea48e8a774579a77a56368"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "c0cbbdfbd3ee24be6900bc6257e21de7"
server_key_iv: "8c3e49b1fe1d5a1a0647e76949ea391d"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.826089 26349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:26:35.829909 26372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.831357 26349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:26:35.831616 26349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal
uuid: "b338b9407bea48e8a774579a77a56368"
format_stamp: "Formatted at 2026-08-07 18:26:35 on dist-test-slave-sb2z"
server_key: "c0cbbdfbd3ee24be6900bc6257e21de7"
server_key_iv: "8c3e49b1fe1d5a1a0647e76949ea391d"
server_key_version: "encryptionkey@0"
I20260807 18:26:35.831877 26349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:35.861488 26349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:35.862432 26349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:35.862773 26349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:35.863904 26349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:35.865512 26349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:35.865656 26349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.865803 26349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:35.865897 26349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:35.901750 26349 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.194:37177
I20260807 18:26:35.901837 26527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.194:37177 every 8 connection(s)
I20260807 18:26:35.903200 26349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/data/info.pb
I20260807 18:26:35.907346 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26349
I20260807 18:26:35.907584 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal/instance
I20260807 18:26:35.908174 30323 external_mini_cluster.cc:1428] Setting key eae197d1f9c40e94432a96487dc837cd
I20260807 18:26:35.911355 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.156.195:0
--local_ip_for_outbound_sockets=127.29.156.195
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--builtin_ntp_servers=127.29.156.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:26:35.917382 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:35.906137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.194:49205 (local address 127.29.156.254:43563)
0807 18:26:35.906589 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:35.906593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:35.908179 (+ 1586us) server_negotiation.cc:407] Connection header received
0807 18:26:35.910335 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:35.910340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:35.910433 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:35.910561 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:35.911943 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.912700 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.914295 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.914710 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.914856 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:35.916403 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:35.916409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:35.916412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:35.916432 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:35.916475 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:35.916487 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:35.916606 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:35.916862 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 18:26:35.916963 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260807 18:26:35.918414 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:35.906748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43563 (local address 127.29.156.194:49205)
0807 18:26:35.907984 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:35.908030 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:35.910037 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:35.910876 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:35.910906 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:35.911020 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:35.911733 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:35.911758 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.912878 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.912883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:35.914142 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:35.914155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:35.914963 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:35.914971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:35.915218 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:35.916008 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:35.916011 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:35.916057 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:35.916186 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:35.916605 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:35.916618 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:35.916629 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:26:35.916832 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 18:26:35.917260 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":440,"threads_started":1}
I20260807 18:26:35.919975 26528 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43563
I20260807 18:26:35.920308 26528 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:35.921164 26528 heartbeater.cc:511] Master 127.29.156.254:43563 requested a full tablet report, sending...
I20260807 18:26:35.922755 26086 ts_manager.cc:194] Registered new tserver with Master: b338b9407bea48e8a774579a77a56368 (127.29.156.194:37177)
I20260807 18:26:35.924048 26086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.194:49205
W20260807 18:26:36.263621 26532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:36.263942 26532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:36.264093 26532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:36.271498 26532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:36.271632 26532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:36.271809 26532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.156.195
I20260807 18:26:36.281091 26532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.156.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.156.195
--webserver_port=0
--tserver_master_addrs=127.29.156.254:43563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.156.195
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:36.282449 26532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:36.284361 26532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:36.298120 26541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:36.298175 26545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:36.298231 26538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:36.299804 26532 server_base.cc:1034] running on GCE node
I20260807 18:26:36.300519 26532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:36.301544 26532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:36.302820 26532 hybrid_clock.cc:648] HybridClock initialized: now 1786127196302759 us; error 47 us; skew 500 ppm
I20260807 18:26:36.303249 26532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:26:36.306025 26532 webserver.cc:460] Webserver started at http://127.29.156.195:33527/ using document root <none> and password file <none>
I20260807 18:26:36.306784 26532 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:36.306905 26532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:36.307257 26532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:36.310923 26532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data/instance:
uuid: "58a764ed6a2d44c09e3b13a676eca1a7"
format_stamp: "Formatted at 2026-08-07 18:26:36 on dist-test-slave-sb2z"
server_key: "3f9f5635a89e587124ff8f6f70babe0f"
server_key_iv: "8e86f6179d2eba1d0ab8c8a412d55f6e"
server_key_version: "encryptionkey@0"
I20260807 18:26:36.311864 26532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance:
uuid: "58a764ed6a2d44c09e3b13a676eca1a7"
format_stamp: "Formatted at 2026-08-07 18:26:36 on dist-test-slave-sb2z"
server_key: "3f9f5635a89e587124ff8f6f70babe0f"
server_key_iv: "8e86f6179d2eba1d0ab8c8a412d55f6e"
server_key_version: "encryptionkey@0"
I20260807 18:26:36.317167 26532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:26:36.320678 26554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:36.322340 26532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:26:36.322554 26532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal
uuid: "58a764ed6a2d44c09e3b13a676eca1a7"
format_stamp: "Formatted at 2026-08-07 18:26:36 on dist-test-slave-sb2z"
server_key: "3f9f5635a89e587124ff8f6f70babe0f"
server_key_iv: "8e86f6179d2eba1d0ab8c8a412d55f6e"
server_key_version: "encryptionkey@0"
I20260807 18:26:36.322790 26532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:36.353833 26532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:36.354735 26532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:36.355063 26532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:36.356215 26532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:26:36.357764 26532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:26:36.357892 26532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:36.358039 26532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:26:36.358129 26532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:36.394773 26532 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.195:38881
I20260807 18:26:36.394866 26715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.195:38881 every 8 connection(s)
I20260807 18:26:36.396135 26532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/data/info.pb
I20260807 18:26:36.397974 26341 heartbeater.cc:503] Master 127.29.156.254:43563 was elected leader, sending a full tablet report...
I20260807 18:26:36.398669 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26532
I20260807 18:26:36.398885 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal/instance
I20260807 18:26:36.399590 30323 external_mini_cluster.cc:1428] Setting key 15b57c1f82b4725b0ed5a5455a909425
I20260807 18:26:36.410282 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.399161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:58565 (local address 127.29.156.254:43563)
0807 18:26:36.399764 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.399771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.401516 (+ 1745us) server_negotiation.cc:407] Connection header received
0807 18:26:36.403846 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.403852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.403935 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.404061 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:36.405169 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.405912 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.407483 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.407927 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.408068 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.409466 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:36.409471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:36.409475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:36.409493 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:36.409529 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:36.409541 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:36.409642 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.409873 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.409972 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476}
I20260807 18:26:36.411389 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.400091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:43563 (local address 127.29.156.195:58565)
0807 18:26:36.401353 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.401408 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.403505 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.404309 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.404318 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.404405 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:26:36.404981 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.404998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.406075 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.406081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.407322 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.407334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.408160 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.408167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.408417 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.409130 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:26:36.409132 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:26:36.409171 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:26:36.409245 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:26:36.409643 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:26:36.409657 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:26:36.409667 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.409861 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.410288 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":362,"threads_started":1}
I20260807 18:26:36.412977 26716 heartbeater.cc:347] Connected to a master server at 127.29.156.254:43563
I20260807 18:26:36.413324 26716 heartbeater.cc:464] Registering TS with master...
I20260807 18:26:36.414237 26716 heartbeater.cc:511] Master 127.29.156.254:43563 requested a full tablet report, sending...
I20260807 18:26:36.415843 26086 ts_manager.cc:194] Registered new tserver with Master: 58a764ed6a2d44c09e3b13a676eca1a7 (127.29.156.195:38881)
I20260807 18:26:36.416749 26086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.156.195:58565
I20260807 18:26:36.428287 30323 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:26:36.444284 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.437532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.29.156.254:43563)
0807 18:26:36.437737 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.437741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.438266 (+ 525us) server_negotiation.cc:407] Connection header received
0807 18:26:36.438449 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.438453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.438520 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.438611 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:36.439216 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.440197 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.441666 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.442084 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.442207 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.443519 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:36.443585 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:36.443592 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:36.443617 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:36.443657 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:36.443671 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:36.443745 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.443914 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.444008 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:26:36.448763 26086 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:26:36.456288 26086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:26:36.483197 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.471022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.29.156.193:34101)
0807 18:26:36.473673 (+ 2651us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.473682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.473715 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:26:36.473818 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.473828 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.474114 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.474325 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:36.475387 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.477098 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.480062 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.481862 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.482092 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.482339 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.482655 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.482833 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2432,"thread_start_us":2581,"threads_started":1}
I20260807 18:26:36.483197 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.470443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48604)
0807 18:26:36.472854 (+ 2411us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.472872 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.473053 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.474637 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.474645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.474677 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:36.475130 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.475142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.477276 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.477284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.479806 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.479822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.482085 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.482094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.482361 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.482432 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.482589 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.482807 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2277,"mutex_wait_us":1,"thread_start_us":2118,"threads_started":1}
I20260807 18:26:36.484897 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.470340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.29.156.194:37177)
0807 18:26:36.474043 (+ 3703us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.474051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.474081 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:26:36.474419 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.474428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.474648 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.474855 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:36.475812 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.478505 (+ 2693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.481799 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.483659 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.483883 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.484106 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.484282 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.484482 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3557,"thread_start_us":1364,"threads_started":1}
I20260807 18:26:36.485419 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.470294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:37177 (local address 127.0.0.1:53338)
0807 18:26:36.471401 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.471469 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.474204 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.475122 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.475138 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.475164 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:36.475600 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.475615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.478683 (+ 3068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.478688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.481577 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.481593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.483916 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.483923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.484085 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.484116 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.484211 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.484332 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":381,"thread_start_us":643,"threads_started":1}
I20260807 18:26:36.487599 26256 tablet_service.cc:1467] Processing CreateTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f (DEFAULT_TABLE table=test-table [id=ad56f761dd0641dfbdf0592f6d1e8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:36.489053 26256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c7958c6f1d4a08b63a6f17e019534f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:36.489403 26434 tablet_service.cc:1467] Processing CreateTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f (DEFAULT_TABLE table=test-table [id=ad56f761dd0641dfbdf0592f6d1e8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:36.490799 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.471910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.195:38881 (local address 127.0.0.1:38484)
0807 18:26:36.480493 (+ 8583us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.480515 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.480641 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.481804 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.481810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.481837 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:36.482289 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.482304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.484500 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.484507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.487857 (+ 3350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.487874 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.489900 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.489910 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.490200 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.490234 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.490341 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.490484 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8449,"thread_start_us":8281,"threads_started":1}
I20260807 18:26:36.490975 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c7958c6f1d4a08b63a6f17e019534f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:36.491051 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.472408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.29.156.195:38881)
0807 18:26:36.480929 (+ 8521us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.480939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.480973 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:26:36.481087 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.481095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.481348 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.481564 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:36.482524 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.484295 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.488086 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.489676 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.489909 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.490168 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.490416 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.490615 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7077,"thread_start_us":6887,"threads_started":1}
I20260807 18:26:36.497299 26620 tablet_service.cc:1467] Processing CreateTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f (DEFAULT_TABLE table=test-table [id=ad56f761dd0641dfbdf0592f6d1e8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:26:36.498929 26620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c7958c6f1d4a08b63a6f17e019534f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:36.501050 26740 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Bootstrap starting.
I20260807 18:26:36.504292 26740 tablet_bootstrap.cc:654] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:36.505306 26740 log.cc:824] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:36.508579 26742 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Bootstrap starting.
I20260807 18:26:36.508713 26740 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: No bootstrap required, opened a new log
I20260807 18:26:36.508989 26740 ts_tablet_manager.cc:1397] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 18:26:36.513271 26742 tablet_bootstrap.cc:654] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:36.514537 26742 log.cc:824] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:36.515120 26743 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Bootstrap starting.
I20260807 18:26:36.518694 26743 tablet_bootstrap.cc:654] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:36.520011 26743 log.cc:824] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:36.521489 26740 raft_consensus.cc:348] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.522028 26740 raft_consensus.cc:374] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:36.522145 26740 raft_consensus.cc:729] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abf4d7e234c451391b85144259357c5, State: Initialized, Role: FOLLOWER
I20260807 18:26:36.522835 26740 consensus_queue.cc:260] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.524780 26742 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: No bootstrap required, opened a new log
I20260807 18:26:36.524843 26740 ts_tablet_manager.cc:1428] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260807 18:26:36.525079 26742 ts_tablet_manager.cc:1397] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260807 18:26:36.525710 26743 tablet_bootstrap.cc:492] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: No bootstrap required, opened a new log
I20260807 18:26:36.526012 26743 ts_tablet_manager.cc:1397] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 18:26:36.534135 26743 raft_consensus.cc:348] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.534590 26743 raft_consensus.cc:374] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:36.534727 26743 raft_consensus.cc:729] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a764ed6a2d44c09e3b13a676eca1a7, State: Initialized, Role: FOLLOWER
I20260807 18:26:36.535416 26743 consensus_queue.cc:260] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.535678 26742 raft_consensus.cc:348] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.536198 26742 raft_consensus.cc:374] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:36.536326 26742 raft_consensus.cc:729] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b338b9407bea48e8a774579a77a56368, State: Initialized, Role: FOLLOWER
I20260807 18:26:36.536686 26716 heartbeater.cc:503] Master 127.29.156.254:43563 was elected leader, sending a full tablet report...
I20260807 18:26:36.536953 26742 consensus_queue.cc:260] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.537356 26743 ts_tablet_manager.cc:1428] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 18:26:36.538306 26528 heartbeater.cc:503] Master 127.29.156.254:43563 was elected leader, sending a full tablet report...
I20260807 18:26:36.538996 26742 ts_tablet_manager.cc:1428] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
W20260807 18:26:36.622537 26345 tablet.cc:2369] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:36.648046 26717 tablet.cc:2369] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:26:36.655624 26529 tablet.cc:2369] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:26:36.804543 26750 raft_consensus.cc:482] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:26:36.804909 26750 raft_consensus.cc:504] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.806821 26750 leader_election.cc:288] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b338b9407bea48e8a774579a77a56368 (127.29.156.194:37177), 9abf4d7e234c451391b85144259357c5 (127.29.156.193:34101)
I20260807 18:26:36.815336 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.807427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:47169 (local address 127.29.156.194:37177)
0807 18:26:36.807677 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.807685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.807799 (+ 114us) server_negotiation.cc:407] Connection header received
0807 18:26:36.808152 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.808160 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.808262 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.808345 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:36.808790 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.809802 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.812581 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.814030 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.814205 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.814284 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.814899 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.815039 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 18:26:36.815323 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.807462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.156.195:39167 (local address 127.29.156.193:34101)
0807 18:26:36.807736 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.807744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.808408 (+ 664us) server_negotiation.cc:407] Connection header received
0807 18:26:36.808589 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.808592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.808666 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.808773 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:26:36.809268 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.810198 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.812796 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.814036 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.814214 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.814288 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.814891 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.815034 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 18:26:36.815347 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.807304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.194:37177 (local address 127.29.156.195:47169)
0807 18:26:36.807678 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.807698 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.807888 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.808419 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.808424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.808445 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:36.808640 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.808649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.809952 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.810011 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.812393 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.812411 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.814230 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.814237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.814544 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.814705 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.814807 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.814992 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":145}
I20260807 18:26:36.815403 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.807426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.29.156.195:39167)
0807 18:26:36.808272 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:36.808286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:26:36.808433 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:26:36.808809 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:26:36.808811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:26:36.808831 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:26:36.809119 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.809127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.810331 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.810338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.812506 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:26:36.812521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.814235 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.814239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:26:36.814540 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.814705 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 18:26:36.814807 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:26:36.814992 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":180,"thread_start_us":510,"threads_started":1}
I20260807 18:26:36.816663 26284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c7958c6f1d4a08b63a6f17e019534f" candidate_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abf4d7e234c451391b85144259357c5" is_pre_election: true
I20260807 18:26:36.816819 26462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c7958c6f1d4a08b63a6f17e019534f" candidate_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338b9407bea48e8a774579a77a56368" is_pre_election: true
I20260807 18:26:36.817345 26284 raft_consensus.cc:2454] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a764ed6a2d44c09e3b13a676eca1a7 in term 0.
I20260807 18:26:36.817561 26462 raft_consensus.cc:2454] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a764ed6a2d44c09e3b13a676eca1a7 in term 0.
I20260807 18:26:36.818122 26557 leader_election.cc:302] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a764ed6a2d44c09e3b13a676eca1a7, 9abf4d7e234c451391b85144259357c5; no voters:
I20260807 18:26:36.818557 26750 raft_consensus.cc:2781] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:26:36.818692 26750 raft_consensus.cc:482] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:26:36.818835 26750 raft_consensus.cc:3037] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:36.821254 26750 raft_consensus.cc:504] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.821918 26750 leader_election.cc:288] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [CANDIDATE]: Term 1 election: Requested vote from peers b338b9407bea48e8a774579a77a56368 (127.29.156.194:37177), 9abf4d7e234c451391b85144259357c5 (127.29.156.193:34101)
I20260807 18:26:36.822383 26462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c7958c6f1d4a08b63a6f17e019534f" candidate_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338b9407bea48e8a774579a77a56368"
I20260807 18:26:36.822453 26284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c7958c6f1d4a08b63a6f17e019534f" candidate_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abf4d7e234c451391b85144259357c5"
I20260807 18:26:36.822587 26462 raft_consensus.cc:3037] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:36.822706 26284 raft_consensus.cc:3037] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:36.825206 26462 raft_consensus.cc:2454] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a764ed6a2d44c09e3b13a676eca1a7 in term 1.
I20260807 18:26:36.825505 26284 raft_consensus.cc:2454] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a764ed6a2d44c09e3b13a676eca1a7 in term 1.
I20260807 18:26:36.825762 26555 leader_election.cc:302] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a764ed6a2d44c09e3b13a676eca1a7, b338b9407bea48e8a774579a77a56368; no voters:
I20260807 18:26:36.826112 26750 raft_consensus.cc:2781] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:36.826606 26750 raft_consensus.cc:686] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 1 LEADER]: Becoming Leader. State: Replica: 58a764ed6a2d44c09e3b13a676eca1a7, State: Running, Role: LEADER
I20260807 18:26:36.827070 26750 consensus_queue.cc:237] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } }
I20260807 18:26:36.831784 26086 catalog_manager.cc:5495] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58a764ed6a2d44c09e3b13a676eca1a7 (127.29.156.195). New cstate: current_term: 1 leader_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a764ed6a2d44c09e3b13a676eca1a7" member_type: VOTER last_known_addr { host: "127.29.156.195" port: 38881 } health_report { overall_health: HEALTHY } } }
I20260807 18:26:36.848479 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.842790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.29.156.195:38881)
0807 18:26:36.843010 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.843015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.843034 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:26:36.843265 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.843269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.843352 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.843479 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:26:36.843879 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.844599 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.846033 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.846529 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.846685 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.847202 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:26:36.847838 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:26:36.848012 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.848108 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.848194 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:26:36.856988 26462 raft_consensus.cc:1261] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 1 FOLLOWER]: Refusing update from remote peer 58a764ed6a2d44c09e3b13a676eca1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:36.856988 26284 raft_consensus.cc:1261] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 1 FOLLOWER]: Refusing update from remote peer 58a764ed6a2d44c09e3b13a676eca1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:26:36.858098 26754 consensus_queue.cc:1035] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b338b9407bea48e8a774579a77a56368" member_type: VOTER last_known_addr { host: "127.29.156.194" port: 37177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:36.858444 26750 consensus_queue.cc:1035] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abf4d7e234c451391b85144259357c5" member_type: VOTER last_known_addr { host: "127.29.156.193" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:26:36.871061 26757 mvcc.cc:205] Tried to move back new op lower bound from 7315976998315364352 to 7315976998207913984. Current Snapshot: MvccSnapshot[applied={T|T < 7315976998315364352}]
I20260807 18:26:36.871639 26759 mvcc.cc:205] Tried to move back new op lower bound from 7315976998315364352 to 7315976998207913984. Current Snapshot: MvccSnapshot[applied={T|T < 7315976998315364352}]
I20260807 18:26:36.884157 26084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:36.884464 26084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:26:36.888517 26084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccf078a5424844468464dde574bc0d86: Sending DeleteTablet for 3 replicas of tablet b9c7958c6f1d4a08b63a6f17e019534f
I20260807 18:26:36.889792 26256 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:36 UTC) from {username='slave'} at 127.0.0.1:48604
I20260807 18:26:36.889827 26434 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:36 UTC) from {username='slave'} at 127.0.0.1:53338
I20260807 18:26:36.890529 26620 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c7958c6f1d4a08b63a6f17e019534f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:26:36 UTC) from {username='slave'} at 127.0.0.1:38484
I20260807 18:26:36.892057 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26164
I20260807 18:26:36.892812 26768 tablet_replica.cc:330] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: stopping tablet replica
I20260807 18:26:36.893440 26768 raft_consensus.cc:2229] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:36.894179 26768 raft_consensus.cc:2258] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:36.902701 26771 tablet_replica.cc:330] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: stopping tablet replica
I20260807 18:26:36.903182 26771 raft_consensus.cc:2229] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:26:36.904441 26771 raft_consensus.cc:2258] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:36.906174 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:36.893230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48610 (local address 127.29.156.193:34101)
0807 18:26:36.893427 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:36.893436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:36.895413 (+ 1977us) server_negotiation.cc:407] Connection header received
0807 18:26:36.895964 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:36.895974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:36.896119 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:36.896216 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:36.897123 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.898513 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.901693 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:36.902310 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:36.902405 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:36.904333 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:36.904345 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:36.904361 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:36.905106 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:36.905171 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:36.905197 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:36.905299 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:36.905619 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:26:36.905771 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:26:36.906983 26769 tablet_replica.cc:330] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: stopping tablet replica
I20260807 18:26:36.907264 26771 ts_tablet_manager.cc:1905] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:36.907601 26335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:36.907564 26769 raft_consensus.cc:2229] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:26:36.908171 26769 raft_consensus.cc:2258] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:26:36.910243 26769 ts_tablet_manager.cc:1905] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:36.917176 26771 ts_tablet_manager.cc:1918] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:36.917394 26771 log.cc:1196] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-2/wal/wals/b9c7958c6f1d4a08b63a6f17e019534f
I20260807 18:26:36.917785 26769 ts_tablet_manager.cc:1918] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:26:36.918027 26769 log.cc:1196] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Deleting WAL directory at /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126848590636-30323-0/minicluster-data/ts-1/wal/wals/b9c7958c6f1d4a08b63a6f17e019534f
I20260807 18:26:36.918056 26771 ts_tablet_manager.cc:1939] T b9c7958c6f1d4a08b63a6f17e019534f P 58a764ed6a2d44c09e3b13a676eca1a7: Deleting consensus metadata
I20260807 18:26:36.918689 26769 ts_tablet_manager.cc:1939] T b9c7958c6f1d4a08b63a6f17e019534f P b338b9407bea48e8a774579a77a56368: Deleting consensus metadata
I20260807 18:26:36.920320 26063 catalog_manager.cc:4841] TS 58a764ed6a2d44c09e3b13a676eca1a7 (127.29.156.195:38881): tablet b9c7958c6f1d4a08b63a6f17e019534f (table test-table [id=ad56f761dd0641dfbdf0592f6d1e8adf]) successfully deleted
I20260807 18:26:36.920809 26063 catalog_manager.cc:4841] TS b338b9407bea48e8a774579a77a56368 (127.29.156.194:37177): tablet b9c7958c6f1d4a08b63a6f17e019534f (table test-table [id=ad56f761dd0641dfbdf0592f6d1e8adf]) successfully deleted
I20260807 18:26:36.906821 26768 ts_tablet_manager.cc:1905] T b9c7958c6f1d4a08b63a6f17e019534f P 9abf4d7e234c451391b85144259357c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:26:37.014165 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26164
I20260807 18:26:37.049047 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26349
W20260807 18:26:37.049172 26065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.156.193:34101 (error 108)
I20260807 18:26:37.051859 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.050886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48612)
0807 18:26:37.051474 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.051646 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":309,"threads_started":1}
W20260807 18:26:37.052152 26065 catalog_manager.cc:4584] TS 9abf4d7e234c451391b85144259357c5 (127.29.156.193:34101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9c7958c6f1d4a08b63a6f17e019534f: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
I20260807 18:26:37.057731 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.050303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.29.156.194:37177)
0807 18:26:37.050520 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:37.050526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:37.050551 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:26:37.050716 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:37.050723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:37.050844 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:37.050949 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:37.051511 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:37.052603 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:37.054447 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:37.054879 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:37.055034 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:37.056316 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:37.056325 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:37.056339 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:37.056933 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:37.056989 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:37.057009 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:37.057098 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:37.057265 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:26:37.057359 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:26:37.058405 26522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:37.105948 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.105338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48614)
0807 18:26:37.105623 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.105759 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:26:37.107362 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.106863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48628)
0807 18:26:37.107075 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.107173 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:26:37.153251 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26349
I20260807 18:26:37.168381 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.167756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48640)
0807 18:26:37.168033 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.168159 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:26:37.169727 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.169252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48654)
0807 18:26:37.169440 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.169543 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:26:37.179836 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26532
I20260807 18:26:37.188215 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.180681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.29.156.195:38881)
0807 18:26:37.180856 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:37.180860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:37.180953 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:26:37.181230 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:37.181235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:37.181352 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:37.181469 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:37.182020 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:37.182954 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:37.184805 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:37.185275 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:37.185377 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:37.186689 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:37.186698 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:37.186716 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:37.187311 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:37.187368 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:37.187388 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:37.187479 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:37.187707 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 18:26:37.187807 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:26:37.189034 26709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:37.250383 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.249858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48656)
0807 18:26:37.250118 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.250219 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:26:37.251710 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.251202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48670)
0807 18:26:37.251408 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.251489 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:26:37.314486 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26532
I20260807 18:26:37.343457 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26042
I20260807 18:26:37.344852 26125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:37.458841 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:37.457179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.193:34101 (local address 127.0.0.1:48672)
0807 18:26:37.458417 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0807 18:26:37.458589 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":944}
I20260807 18:26:37.459089 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26042
2026-08-07T18:26:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3258 ms)
[----------] 6 tests from EncryptionPolicyTest (25422 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:37 dist-test-slave-sb2z krb5kdc[26784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:37 dist-test-slave-sb2z krb5kdc[26784](info): set up 2 sockets
Aug 07 18:26:37 dist-test-slave-sb2z krb5kdc[26784](info): commencing operation
krb5kdc: starting...
W20260807 18:26:38.868731 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:39 dist-test-slave-sb2z krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:39.236344 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:39.590754 26801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:39.591055 26801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:39.591162 26801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:39.598388 26801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:39.598516 26801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:39.598565 26801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:39.598636 26801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:39.598682 26801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:39.608855 26801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39117
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:39.610265 26801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:39.611690 26801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:39.624652 26813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:39.624625 26808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:39.624706 26807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:39.625566 26801 server_base.cc:1034] running on GCE node
I20260807 18:26:39.626525 26801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:39.628182 26801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:39.629467 26801 hybrid_clock.cc:648] HybridClock initialized: now 1786127199629420 us; error 68 us; skew 500 ppm
I20260807 18:26:39.629877 26801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:39 dist-test-slave-sb2z krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:39.633493 26801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:39.635186 26801 webserver.cc:460] Webserver started at http://127.0.0.1:42181/ using document root <none> and password file <none>
I20260807 18:26:39.636044 26801 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:39.636230 26801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:39.636566 26801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:39.639885 26801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "6e53042bd5084386aef96f0726f7a298"
format_stamp: "Formatted at 2026-08-07 18:26:39 on dist-test-slave-sb2z"
server_key: "ba83d39d9502a603f1c4207e3f7560ef"
server_key_iv: "cee00e038043054d03340131de4d309d"
server_key_version: "encryptionkey@0"
I20260807 18:26:39.640796 26801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "6e53042bd5084386aef96f0726f7a298"
format_stamp: "Formatted at 2026-08-07 18:26:39 on dist-test-slave-sb2z"
server_key: "ba83d39d9502a603f1c4207e3f7560ef"
server_key_iv: "cee00e038043054d03340131de4d309d"
server_key_version: "encryptionkey@0"
I20260807 18:26:39.646425 26801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:26:39.650327 26825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:39.651875 26801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:26:39.652076 26801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "6e53042bd5084386aef96f0726f7a298"
format_stamp: "Formatted at 2026-08-07 18:26:39 on dist-test-slave-sb2z"
server_key: "ba83d39d9502a603f1c4207e3f7560ef"
server_key_iv: "cee00e038043054d03340131de4d309d"
server_key_version: "encryptionkey@0"
I20260807 18:26:39.652292 26801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:39.672602 26801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:39.677274 26801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:39.677609 26801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:39.698638 26801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39117
I20260807 18:26:39.698678 26898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39117 every 8 connection(s)
I20260807 18:26:39.700194 26801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:39.701895 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26801
I20260807 18:26:39.702140 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:39.703011 30323 external_mini_cluster.cc:1428] Setting key 90a9f9b7bf288c29dbee0a54155f4ac5
I20260807 18:26:39.705261 26900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:26:39 dist-test-slave-sb2z krb5kdc[26784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:26:39.719754 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298: Bootstrap starting.
I20260807 18:26:39.724494 26900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:39.725826 26900 log.cc:824] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:39.729636 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298: No bootstrap required, opened a new log
I20260807 18:26:39.729869 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:39.705107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55355 (local address 127.0.0.1:39117)
0807 18:26:39.705943 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:39.705954 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:39.705998 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:26:39.707656 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:39.707678 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:39.708144 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:39.708479 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:39.710540 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:39.712005 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:39.713840 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:39.714536 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:39.714686 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:39.719923 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:39.719936 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:39.719957 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:39.720012 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:39.724418 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:39.725490 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:39.725500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:39.725510 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:39.725774 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:39.726492 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:39.726501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:39.726507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:39.727200 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:39.727552 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:39.728116 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 18:26:39.728477 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":420,"threads_started":1}
I20260807 18:26:39.734491 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:39.738797 26900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260807 18:26:39.739241 26900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:39.739372 26900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e53042bd5084386aef96f0726f7a298, State: Initialized, Role: FOLLOWER
I20260807 18:26:39.740280 26900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260807 18:26:39.740585 26900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:39.740767 26900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:39.741001 26900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:39.743927 26900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260807 18:26:39.744491 26900 leader_election.cc:302] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e53042bd5084386aef96f0726f7a298; no voters:
I20260807 18:26:39.745177 26900 leader_election.cc:288] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:39.745337 26912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:39.746711 26912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [term 1 LEADER]: Becoming Leader. State: Replica: 6e53042bd5084386aef96f0726f7a298, State: Running, Role: LEADER
I20260807 18:26:39.747457 26912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260807 18:26:39.747877 26900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:39.750591 26915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e53042bd5084386aef96f0726f7a298. Latest consensus state: current_term: 1 leader_uuid: "6e53042bd5084386aef96f0726f7a298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } } }
I20260807 18:26:39.750603 26913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e53042bd5084386aef96f0726f7a298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e53042bd5084386aef96f0726f7a298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } } }
I20260807 18:26:39.751070 26913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:39.751070 26915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:39.761961 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:39.739723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39953 (local address 127.0.0.1:39117)
0807 18:26:39.740007 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:39.740015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:39.740477 (+ 462us) server_negotiation.cc:407] Connection header received
0807 18:26:39.740703 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:39.740709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:39.740804 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:39.740938 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:39.742516 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:39.743745 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:39.745426 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:39.746209 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:39.747342 (+ 1133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:39.750022 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:39.750029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:39.750037 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:39.750096 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:39.755150 (+ 5054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:39.756879 (+ 1729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:39.756887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:39.757013 (+ 126us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:39.757374 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:39.758154 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:39.758163 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:39.758169 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:39.758691 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:39.760801 (+ 2110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:39.761102 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:26:39.761425 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:26:39.765707 26922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:39.771979 26922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:39.781591 26922 catalog_manager.cc:1347] Generated new cluster ID: 080959e7c79a4decabf08f6ee0c5ffe9
I20260807 18:26:39.781725 26922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:39.809671 26922 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:39.811132 26922 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:39.818229 26922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e53042bd5084386aef96f0726f7a298: Generated new TSK 0
I20260807 18:26:39.819056 26922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:39.831918 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26801
I20260807 18:26:39.833456 26894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:39.930249 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26801
W20260807 18:26:39.953272 26782 connection.cc:535] client connection to 127.0.0.1:39117 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39117): Connection reset by peer (error 104)
2026-08-07T18:26:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:40 dist-test-slave-sb2z krb5kdc[26945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:40 dist-test-slave-sb2z krb5kdc[26945](info): set up 2 sockets
Aug 07 18:26:40 dist-test-slave-sb2z krb5kdc[26945](info): commencing operation
krb5kdc: starting...
W20260807 18:26:41.256551 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:41 dist-test-slave-sb2z krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:41.615921 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:41.981418 26966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:41.981720 26966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:41.981904 26966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:41.989092 26966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:41.989200 26966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:41.989295 26966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:41.989384 26966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:41.989473 26966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:41.999271 26966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:42.000670 26966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:42.002040 26966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:42.014483 26975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:42.014536 26977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:42.014700 26979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:42.015666 26966 server_base.cc:1034] running on GCE node
I20260807 18:26:42.016531 26966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:42.018146 26966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:42.019413 26966 hybrid_clock.cc:648] HybridClock initialized: now 1786127202019383 us; error 57 us; skew 500 ppm
I20260807 18:26:42.019855 26966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:42 dist-test-slave-sb2z krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:42.023327 26966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:42.025099 26966 webserver.cc:460] Webserver started at http://127.0.0.1:38459/ using document root <none> and password file <none>
I20260807 18:26:42.025769 26966 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:42.025879 26966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:42.026154 26966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:42.029320 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "fd5364803462457a866f1cc5a479287c"
format_stamp: "Formatted at 2026-08-07 18:26:42 on dist-test-slave-sb2z"
server_key: "d59cb429d1e69e8a83a03ace1ce71bdb"
server_key_iv: "abd610f6b3631a1724f263337b234865"
server_key_version: "encryptionkey@0"
I20260807 18:26:42.030067 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "fd5364803462457a866f1cc5a479287c"
format_stamp: "Formatted at 2026-08-07 18:26:42 on dist-test-slave-sb2z"
server_key: "d59cb429d1e69e8a83a03ace1ce71bdb"
server_key_iv: "abd610f6b3631a1724f263337b234865"
server_key_version: "encryptionkey@0"
I20260807 18:26:42.035636 26966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:26:42.039261 26988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:42.041019 26966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:26:42.041218 26966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "fd5364803462457a866f1cc5a479287c"
format_stamp: "Formatted at 2026-08-07 18:26:42 on dist-test-slave-sb2z"
server_key: "d59cb429d1e69e8a83a03ace1ce71bdb"
server_key_iv: "abd610f6b3631a1724f263337b234865"
server_key_version: "encryptionkey@0"
I20260807 18:26:42.041405 26966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:42.066111 26966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:42.072201 26966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:42.072502 26966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:42.091390 26966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42205
I20260807 18:26:42.091430 27059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42205 every 8 connection(s)
I20260807 18:26:42.092852 26966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:42.097158 27060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:42.101655 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 26966
I20260807 18:26:42.101878 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:42.102350 30323 external_mini_cluster.cc:1428] Setting key ffb69e03fbccb4a0a98a10e436cd31f1
I20260807 18:26:42.112708 27060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: Bootstrap starting.
I20260807 18:26:42.117296 27060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:42.118824 27060 log.cc:824] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:42.118961 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:42.104417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46379 (local address 127.0.0.1:42205)
0807 18:26:42.105352 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:42.105369 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:42.105430 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:26:42.107415 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:42.107451 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:42.108124 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:42.108675 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:42.109293 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:42.111630 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:42.113148 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:42.114277 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:42.114486 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:42.115758 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:42.115773 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:42.115798 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:42.116616 (+ 818us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:42.116720 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:42.116759 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:42.116947 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:42.117229 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:26:42.117440 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":378,"threads_started":1}
I20260807 18:26:42.122648 27060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: No bootstrap required, opened a new log
I20260807 18:26:42.123339 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:42.131357 27060 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260807 18:26:42.131897 27060 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:42.132015 27060 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5364803462457a866f1cc5a479287c, State: Initialized, Role: FOLLOWER
I20260807 18:26:42.132712 27060 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260807 18:26:42.132979 27060 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:42.133153 27060 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:42.133343 27060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:42.135701 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:42.128160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59573 (local address 127.0.0.1:42205)
0807 18:26:42.128403 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:42.128408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:42.128765 (+ 357us) server_negotiation.cc:407] Connection header received
0807 18:26:42.128983 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:42.128986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:42.129060 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:42.129182 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:42.129819 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:42.130605 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:42.132559 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:42.133183 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:42.133351 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:42.134724 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:42.134729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:42.134732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:42.134750 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:42.134788 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:42.134807 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:42.134898 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:42.135203 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 18:26:42.135273 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:26:42.136488 27060 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260807 18:26:42.137022 27060 leader_election.cc:302] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd5364803462457a866f1cc5a479287c; no voters:
I20260807 18:26:42.137717 27060 leader_election.cc:288] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:42.137804 27072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:42.138705 27072 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [term 1 LEADER]: Becoming Leader. State: Replica: fd5364803462457a866f1cc5a479287c, State: Running, Role: LEADER
I20260807 18:26:42.139585 27072 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260807 18:26:42.140372 27060 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:42.143172 27075 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd5364803462457a866f1cc5a479287c. Latest consensus state: current_term: 1 leader_uuid: "fd5364803462457a866f1cc5a479287c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } } }
I20260807 18:26:42.143648 27075 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:42.143157 27074 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd5364803462457a866f1cc5a479287c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5364803462457a866f1cc5a479287c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } } }
I20260807 18:26:42.144208 27074 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c [sys.catalog]: This master's current role is: LEADER
W20260807 18:26:42.151422 27100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:26:42.151633 27100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:26:42.151794 27088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:42.153514 27088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:42.162082 27088 catalog_manager.cc:1347] Generated new cluster ID: b55040d114b94644aaf1aef088968770
I20260807 18:26:42.162213 27088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:42.172936 27088 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:42.174228 27088 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:42.190801 27088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd5364803462457a866f1cc5a479287c: Generated new TSK 0
I20260807 18:26:42.191960 27088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:42.205749 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26966
W20260807 18:26:42.206940 27055 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46379
W20260807 18:26:42.207780 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 26966: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:42.207988 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 26966
W20260807 18:26:42.230305 26943 connection.cc:535] client connection to 127.0.0.1:42205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42205): Connection reset by peer (error 104)
2026-08-07T18:26:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:42 dist-test-slave-sb2z krb5kdc[27106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:42 dist-test-slave-sb2z krb5kdc[27106](info): set up 2 sockets
Aug 07 18:26:42 dist-test-slave-sb2z krb5kdc[27106](info): commencing operation
krb5kdc: starting...
W20260807 18:26:43.602785 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:43 dist-test-slave-sb2z krb5kdc[27106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:43.976223 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35727
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:44.348685 27120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:44.349171 27120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:44.349393 27120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:44.357930 27120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:44.358052 27120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:44.358100 27120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:44.358170 27120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:44.358217 27120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:44.368532 27120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35727
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:44.369807 27120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:44.371097 27120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:44.383474 27128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:44.383445 27131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:44.384140 27133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:44.384573 27120 server_base.cc:1034] running on GCE node
I20260807 18:26:44.385423 27120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:44.386904 27120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:44.388160 27120 hybrid_clock.cc:648] HybridClock initialized: now 1786127204388127 us; error 60 us; skew 500 ppm
I20260807 18:26:44.388541 27120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:44 dist-test-slave-sb2z krb5kdc[27106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:44.391919 27120 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:44.393410 27120 webserver.cc:460] Webserver started at http://127.0.0.1:40689/ using document root <none> and password file <none>
I20260807 18:26:44.394122 27120 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:44.394277 27120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:44.394608 27120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:44.397706 27120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "748e7e624cc344a4bbc10f4189f7cbf8"
format_stamp: "Formatted at 2026-08-07 18:26:44 on dist-test-slave-sb2z"
server_key: "cc7bc6df57c10f5f631fb0f4e6339372"
server_key_iv: "24d91cba35240d4d31fe071e80f96c9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:44.398519 27120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "748e7e624cc344a4bbc10f4189f7cbf8"
format_stamp: "Formatted at 2026-08-07 18:26:44 on dist-test-slave-sb2z"
server_key: "cc7bc6df57c10f5f631fb0f4e6339372"
server_key_iv: "24d91cba35240d4d31fe071e80f96c9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:44.403944 27120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:26:44.408003 27141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:44.409322 27120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:26:44.409518 27120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "748e7e624cc344a4bbc10f4189f7cbf8"
format_stamp: "Formatted at 2026-08-07 18:26:44 on dist-test-slave-sb2z"
server_key: "cc7bc6df57c10f5f631fb0f4e6339372"
server_key_iv: "24d91cba35240d4d31fe071e80f96c9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:44.409734 27120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:44.415457 27120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:44.420066 27120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:44.420382 27120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:44.440486 27120 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35727
I20260807 18:26:44.440510 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35727 every 8 connection(s)
I20260807 18:26:44.441982 27120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:44.443099 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27120
I20260807 18:26:44.443404 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:44.444100 30323 external_mini_cluster.cc:1428] Setting key e651ecf57deb257549359adecc19b958
I20260807 18:26:44.447235 27214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:44.452368 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:44.446184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46533 (local address 127.0.0.1:35727)
0807 18:26:44.447003 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:44.447016 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:44.447058 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:26:44.448835 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:44.448859 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:44.449193 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:44.449607 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:44.450691 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:44.450697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:44.450713 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:44.450726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:44.450806 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:44.450825 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:44.451064 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:44.451218 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:26:44.451397 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":360,"threads_started":1}
I20260807 18:26:44.456239 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:44.460107 27214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8: Bootstrap starting.
I20260807 18:26:44.463601 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:44.460965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49997 (local address 127.0.0.1:35727)
0807 18:26:44.461161 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:44.461167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:44.461493 (+ 326us) server_negotiation.cc:407] Connection header received
0807 18:26:44.461631 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:44.461637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:44.461707 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:44.461847 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:44.462861 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:44.462867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:44.462872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:44.462890 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:44.462940 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:44.462960 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:44.463087 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:44.463176 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 18:26:44.463239 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:26:44.464437 27214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:44.465986 27214 log.cc:824] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:44.469343 27214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8: No bootstrap required, opened a new log
I20260807 18:26:44.477775 27214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260807 18:26:44.478168 27214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:44.478255 27214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748e7e624cc344a4bbc10f4189f7cbf8, State: Initialized, Role: FOLLOWER
I20260807 18:26:44.478796 27214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260807 18:26:44.479010 27214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:44.479125 27214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:44.479267 27214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:44.481734 27214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260807 18:26:44.482386 27214 leader_election.cc:302] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 748e7e624cc344a4bbc10f4189f7cbf8; no voters:
I20260807 18:26:44.483103 27214 leader_election.cc:288] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:44.483273 27228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:44.484108 27228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 748e7e624cc344a4bbc10f4189f7cbf8, State: Running, Role: LEADER
I20260807 18:26:44.484683 27228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260807 18:26:44.485512 27214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:44.487769 27231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } } }
I20260807 18:26:44.488279 27231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:44.487769 27232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 748e7e624cc344a4bbc10f4189f7cbf8. Latest consensus state: current_term: 1 leader_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748e7e624cc344a4bbc10f4189f7cbf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } } }
I20260807 18:26:44.489034 27232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:44.490365 27235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:44.496333 27235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:44.506238 27235 catalog_manager.cc:1347] Generated new cluster ID: bc19977382094de89f0260f0b07ffd00
I20260807 18:26:44.506371 27235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:44.524273 27235 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:44.524744 27235 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:44.532141 27235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 748e7e624cc344a4bbc10f4189f7cbf8: Generated new TSK 0
I20260807 18:26:44.533015 27235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:44.546026 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27120
W20260807 18:26:44.547178 27208 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46533
W20260807 18:26:44.547961 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27120: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:44.548193 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27120
2026-08-07T18:26:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:44 dist-test-slave-sb2z krb5kdc[27257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:44 dist-test-slave-sb2z krb5kdc[27257](info): set up 2 sockets
Aug 07 18:26:44 dist-test-slave-sb2z krb5kdc[27257](info): commencing operation
krb5kdc: starting...
W20260807 18:26:45.906292 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:46 dist-test-slave-sb2z krb5kdc[27257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:46.267666 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:46.621099 27273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:46.621464 27273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:46.621649 27273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:46.629050 27273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:46.629247 27273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:46.629369 27273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:46.629485 27273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:46.629575 27273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:46.639909 27273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:46.641386 27273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:46.642820 27273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:46.655275 27282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:46.655331 27283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:46.655342 27285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:46.656289 27273 server_base.cc:1034] running on GCE node
I20260807 18:26:46.657232 27273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:46.658776 27273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:46.660023 27273 hybrid_clock.cc:648] HybridClock initialized: now 1786127206659986 us; error 56 us; skew 500 ppm
I20260807 18:26:46.660449 27273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:46 dist-test-slave-sb2z krb5kdc[27257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:46.663982 27273 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:46.665753 27273 webserver.cc:460] Webserver started at http://127.0.0.1:37033/ using document root <none> and password file <none>
I20260807 18:26:46.666493 27273 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:46.666656 27273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:46.666986 27273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:46.670228 27273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "c80ae5f4cbfa4398a249b49821250d0a"
format_stamp: "Formatted at 2026-08-07 18:26:46 on dist-test-slave-sb2z"
server_key: "6aa4f7526dc6c8d41bbb010ae9f4ce75"
server_key_iv: "41466360ffae7f8c5ff977c08b7841ba"
server_key_version: "encryptionkey@0"
I20260807 18:26:46.671042 27273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "c80ae5f4cbfa4398a249b49821250d0a"
format_stamp: "Formatted at 2026-08-07 18:26:46 on dist-test-slave-sb2z"
server_key: "6aa4f7526dc6c8d41bbb010ae9f4ce75"
server_key_iv: "41466360ffae7f8c5ff977c08b7841ba"
server_key_version: "encryptionkey@0"
I20260807 18:26:46.676810 27273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 18:26:46.680627 27295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:46.682149 27273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:26:46.682344 27273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "c80ae5f4cbfa4398a249b49821250d0a"
format_stamp: "Formatted at 2026-08-07 18:26:46 on dist-test-slave-sb2z"
server_key: "6aa4f7526dc6c8d41bbb010ae9f4ce75"
server_key_iv: "41466360ffae7f8c5ff977c08b7841ba"
server_key_version: "encryptionkey@0"
I20260807 18:26:46.682559 27273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:46.696964 27273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:46.701567 27273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:46.701879 27273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:46.723047 27273 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41345
I20260807 18:26:46.723155 27366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41345 every 8 connection(s)
I20260807 18:26:46.724629 27273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:46.728502 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:46.733543 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27273
I20260807 18:26:46.733776 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:46.734364 30323 external_mini_cluster.cc:1428] Setting key 408edd7847ece2fe31912b20c3dee45f
I20260807 18:26:46.740967 27367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a: Bootstrap starting.
I20260807 18:26:46.744524 27367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:46.745733 27367 log.cc:824] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:46.748947 27367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a: No bootstrap required, opened a new log
Aug 07 18:26:46 dist-test-slave-sb2z krb5kdc[27257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:26:46.757122 27367 raft_consensus.cc:348] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } }
I20260807 18:26:46.757671 27367 raft_consensus.cc:374] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:46.757812 27367 raft_consensus.cc:729] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80ae5f4cbfa4398a249b49821250d0a, State: Initialized, Role: FOLLOWER
I20260807 18:26:46.758487 27367 consensus_queue.cc:260] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } }
I20260807 18:26:46.758664 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:46.736379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38311 (local address 127.0.0.1:41345)
0807 18:26:46.737225 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:46.737238 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:46.737281 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:26:46.739515 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:46.739621 (+ 106us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:46.740076 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:46.740651 (+ 575us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:46.742287 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:46.743741 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:46.745150 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:46.745916 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:46.746153 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:46.750620 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:46.750631 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:46.750647 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:46.750700 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:46.754092 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:46.754900 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:46.754908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:46.754915 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:46.755105 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:46.755551 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:46.755567 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:46.755572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:46.756187 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:46.756528 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:46.757007 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 18:26:46.757462 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":371,"threads_started":1}
I20260807 18:26:46.758790 27367 raft_consensus.cc:388] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:46.758984 27367 raft_consensus.cc:482] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:46.759223 27367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:46.762387 27367 raft_consensus.cc:504] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } }
I20260807 18:26:46.762956 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:46.762970 27367 leader_election.cc:302] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c80ae5f4cbfa4398a249b49821250d0a; no voters:
I20260807 18:26:46.763689 27367 leader_election.cc:288] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:46.763836 27377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:46.765686 27367 sys_catalog.cc:564] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:46.772588 27377 raft_consensus.cc:686] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [term 1 LEADER]: Becoming Leader. State: Replica: c80ae5f4cbfa4398a249b49821250d0a, State: Running, Role: LEADER
I20260807 18:26:46.773533 27377 consensus_queue.cc:237] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } }
I20260807 18:26:46.776914 27379 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c80ae5f4cbfa4398a249b49821250d0a. Latest consensus state: current_term: 1 leader_uuid: "c80ae5f4cbfa4398a249b49821250d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } } }
I20260807 18:26:46.777438 27379 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:46.778105 27378 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c80ae5f4cbfa4398a249b49821250d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ae5f4cbfa4398a249b49821250d0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41345 } } }
I20260807 18:26:46.778631 27378 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:46.787750 27399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:46.794548 27399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:46.806416 27399 catalog_manager.cc:1347] Generated new cluster ID: 96e38a9e34744dd988666dd4b37292ee
I20260807 18:26:46.806651 27399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:46.807930 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:46.776646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41861 (local address 127.0.0.1:41345)
0807 18:26:46.776855 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:46.776862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:46.788305 (+ 11443us) server_negotiation.cc:407] Connection header received
0807 18:26:46.788699 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:46.788708 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:46.788829 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:46.788925 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:46.791953 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:46.793567 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:46.795604 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:46.796360 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:46.796471 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:46.800302 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:46.800312 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:46.800317 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:46.800358 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:46.804344 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:46.805340 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:46.805349 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:46.805355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:46.805513 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:46.806041 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:46.806046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:46.806050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:46.806538 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:46.806840 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:46.807234 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:26:46.807493 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:26:46.831427 27399 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:46.833485 27399 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:46.847625 27399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c80ae5f4cbfa4398a249b49821250d0a: Generated new TSK 0
I20260807 18:26:46.848608 27399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:46.860996 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27273
I20260807 18:26:46.862344 27362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:46.958959 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27273
2026-08-07T18:26:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:47 dist-test-slave-sb2z krb5kdc[27414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:47 dist-test-slave-sb2z krb5kdc[27414](info): set up 2 sockets
Aug 07 18:26:47 dist-test-slave-sb2z krb5kdc[27414](info): commencing operation
krb5kdc: starting...
W20260807 18:26:48.286526 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.004s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:48 dist-test-slave-sb2z krb5kdc[27414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:48.652057 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:49.017798 27430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:49.018126 27430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:49.018224 27430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:49.025647 27430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:49.025785 27430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:49.025833 27430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:49.025959 27430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:49.026015 27430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:49.036232 27430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:49.037792 27430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:49.039167 27430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:49.051725 27440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:49.052021 27438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:49.051760 27442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:49.052647 27430 server_base.cc:1034] running on GCE node
I20260807 18:26:49.053443 27430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:49.054842 27430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:49.056147 27430 hybrid_clock.cc:648] HybridClock initialized: now 1786127209056085 us; error 65 us; skew 500 ppm
I20260807 18:26:49.056557 27430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:49 dist-test-slave-sb2z krb5kdc[27414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:49.059993 27430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:49.061474 27430 webserver.cc:460] Webserver started at http://127.0.0.1:36411/ using document root <none> and password file <none>
I20260807 18:26:49.062135 27430 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:49.062240 27430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:49.062513 27430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:49.065595 27430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "4732fd4034794779bbfb0297e17bf57c"
format_stamp: "Formatted at 2026-08-07 18:26:49 on dist-test-slave-sb2z"
server_key: "f7b89096add6fcb91b54d44a90e43cd1"
server_key_iv: "da3c16a30c78f123fcdaf77192df83d3"
server_key_version: "encryptionkey@0"
I20260807 18:26:49.066349 27430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "4732fd4034794779bbfb0297e17bf57c"
format_stamp: "Formatted at 2026-08-07 18:26:49 on dist-test-slave-sb2z"
server_key: "f7b89096add6fcb91b54d44a90e43cd1"
server_key_iv: "da3c16a30c78f123fcdaf77192df83d3"
server_key_version: "encryptionkey@0"
I20260807 18:26:49.071817 27430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:26:49.075428 27450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:49.076854 27430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:26:49.076998 27430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "4732fd4034794779bbfb0297e17bf57c"
format_stamp: "Formatted at 2026-08-07 18:26:49 on dist-test-slave-sb2z"
server_key: "f7b89096add6fcb91b54d44a90e43cd1"
server_key_iv: "da3c16a30c78f123fcdaf77192df83d3"
server_key_version: "encryptionkey@0"
I20260807 18:26:49.077181 27430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:49.087890 27430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:49.091784 27430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:49.092046 27430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:49.111413 27430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44037
I20260807 18:26:49.111444 27530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44037 every 8 connection(s)
I20260807 18:26:49.112944 27430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:49.116609 27532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:49.120039 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27430
I20260807 18:26:49.120244 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:49.120831 30323 external_mini_cluster.cc:1428] Setting key dd92babc87fcd693317efe60bace16fb
I20260807 18:26:49.129537 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c: Bootstrap starting.
I20260807 18:26:49.133415 27532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:49.134568 27532 log.cc:824] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:49.134768 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:49.122834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35785 (local address 127.0.0.1:44037)
0807 18:26:49.123716 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:49.123729 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:49.123773 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:26:49.125440 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:49.125463 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:49.125878 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:49.126214 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:49.126812 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:49.128232 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:49.130142 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:49.130810 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:49.130953 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:49.132292 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:49.132299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:49.132312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:49.132869 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:49.132941 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:49.132966 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:49.133058 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:49.133311 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 18:26:49.133691 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":430,"threads_started":1}
I20260807 18:26:49.137929 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c: No bootstrap required, opened a new log
I20260807 18:26:49.138829 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:49.148590 27532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260807 18:26:49.149159 27532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:49.149283 27532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4732fd4034794779bbfb0297e17bf57c, State: Initialized, Role: FOLLOWER
I20260807 18:26:49.149991 27532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260807 18:26:49.150260 27532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:49.150419 27532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:49.150624 27532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:49.151858 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:49.144368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42045 (local address 127.0.0.1:44037)
0807 18:26:49.144587 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:49.144594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:49.145054 (+ 460us) server_negotiation.cc:407] Connection header received
0807 18:26:49.145227 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:49.145233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:49.145324 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:49.145436 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:49.146034 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:49.147183 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:49.148797 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:49.149506 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:49.149652 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:49.150942 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:49.150953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:49.150958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:49.150987 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:49.151037 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:49.151062 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:49.151167 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:49.151358 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 18:26:49.151464 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:26:49.153517 27532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260807 18:26:49.154057 27532 leader_election.cc:302] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4732fd4034794779bbfb0297e17bf57c; no voters:
I20260807 18:26:49.154675 27532 leader_election.cc:288] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:49.154855 27546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:49.155776 27546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [term 1 LEADER]: Becoming Leader. State: Replica: 4732fd4034794779bbfb0297e17bf57c, State: Running, Role: LEADER
I20260807 18:26:49.156348 27546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260807 18:26:49.156996 27532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:49.159804 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4732fd4034794779bbfb0297e17bf57c. Latest consensus state: current_term: 1 leader_uuid: "4732fd4034794779bbfb0297e17bf57c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } } }
I20260807 18:26:49.160271 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:49.161844 27555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:49.161729 27547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4732fd4034794779bbfb0297e17bf57c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4732fd4034794779bbfb0297e17bf57c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } } }
I20260807 18:26:49.162052 27547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:49.168100 27555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:49.178177 27555 catalog_manager.cc:1347] Generated new cluster ID: 13a672b591db40be89e3eb06e2d65b7f
I20260807 18:26:49.178318 27555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:49.194123 27555 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:49.195474 27555 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:49.202242 27555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4732fd4034794779bbfb0297e17bf57c: Generated new TSK 0
I20260807 18:26:49.203136 27555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:49.210980 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27430
W20260807 18:26:49.212229 27519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35785
W20260807 18:26:49.212957 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27430: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:49.213137 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27430
2026-08-07T18:26:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:49 dist-test-slave-sb2z krb5kdc[27575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:49 dist-test-slave-sb2z krb5kdc[27575](info): set up 2 sockets
Aug 07 18:26:49 dist-test-slave-sb2z krb5kdc[27575](info): commencing operation
krb5kdc: starting...
W20260807 18:26:50.573871 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:50 dist-test-slave-sb2z krb5kdc[27575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:50.939759 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:51.293478 27589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:51.293876 27589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:51.294087 27589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:51.302210 27589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:51.302342 27589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:51.302420 27589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:51.302522 27589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:51.302593 27589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:51.313133 27589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:51.314492 27589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:51.315930 27589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:51.332193 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:51.332278 27594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:51.332253 27596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:51.333384 27589 server_base.cc:1034] running on GCE node
I20260807 18:26:51.334321 27589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:51.335952 27589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:51.337239 27589 hybrid_clock.cc:648] HybridClock initialized: now 1786127211337180 us; error 45 us; skew 500 ppm
I20260807 18:26:51.337741 27589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:51 dist-test-slave-sb2z krb5kdc[27575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:51.346496 27589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:51.348461 27589 webserver.cc:460] Webserver started at http://127.0.0.1:37329/ using document root <none> and password file <none>
I20260807 18:26:51.349263 27589 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:51.349431 27589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:51.349777 27589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:51.353299 27589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "f5aae779491f41ada7162345278339d6"
format_stamp: "Formatted at 2026-08-07 18:26:51 on dist-test-slave-sb2z"
server_key: "a4d88d086d7822954bbe0163381cb562"
server_key_iv: "26af87032f2143dbb081705570893e9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:51.354254 27589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "f5aae779491f41ada7162345278339d6"
format_stamp: "Formatted at 2026-08-07 18:26:51 on dist-test-slave-sb2z"
server_key: "a4d88d086d7822954bbe0163381cb562"
server_key_iv: "26af87032f2143dbb081705570893e9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:51.360203 27589 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 18:26:51.364070 27609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:51.365545 27589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:26:51.365764 27589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "f5aae779491f41ada7162345278339d6"
format_stamp: "Formatted at 2026-08-07 18:26:51 on dist-test-slave-sb2z"
server_key: "a4d88d086d7822954bbe0163381cb562"
server_key_iv: "26af87032f2143dbb081705570893e9d"
server_key_version: "encryptionkey@0"
I20260807 18:26:51.365995 27589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:51.372640 27589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:51.377336 27589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:51.377686 27589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:51.398844 27589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42443
I20260807 18:26:51.398891 27678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42443 every 8 connection(s)
I20260807 18:26:51.400411 27589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:51.404373 27679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:51.404775 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27589
I20260807 18:26:51.404989 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:51.405496 30323 external_mini_cluster.cc:1428] Setting key 8ef2a722475208bf61942b4912369f48
I20260807 18:26:51.415128 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:51.407475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44043 (local address 127.0.0.1:42443)
0807 18:26:51.408458 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:51.408473 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:51.408530 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:26:51.411236 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:51.411266 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:51.411764 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:51.412219 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:51.413221 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:51.413229 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:51.413245 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:51.413262 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:51.413331 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:51.413361 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:51.413576 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:51.413712 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:26:51.413901 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":365,"threads_started":1}
I20260807 18:26:51.418803 27679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6: Bootstrap starting.
I20260807 18:26:51.419157 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:51.422016 27679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:51.423108 27679 log.cc:824] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:51.426110 27679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6: No bootstrap required, opened a new log
I20260807 18:26:51.426404 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:51.423804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47893 (local address 127.0.0.1:42443)
0807 18:26:51.424063 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:51.424071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:51.424366 (+ 295us) server_negotiation.cc:407] Connection header received
0807 18:26:51.424564 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:51.424570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:51.424631 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:51.424737 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:51.425702 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:51.425707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:51.425712 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:51.425729 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:51.425776 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:51.425790 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:51.425908 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:51.426035 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 18:26:51.426104 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 18:26:51.434010 27679 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260807 18:26:51.434420 27679 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:51.434532 27679 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5aae779491f41ada7162345278339d6, State: Initialized, Role: FOLLOWER
I20260807 18:26:51.435158 27679 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260807 18:26:51.435421 27679 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:51.435599 27679 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:51.435793 27679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:51.438280 27679 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260807 18:26:51.438817 27679 leader_election.cc:302] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5aae779491f41ada7162345278339d6; no voters:
I20260807 18:26:51.439459 27679 leader_election.cc:288] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:51.439656 27691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:51.440455 27691 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [term 1 LEADER]: Becoming Leader. State: Replica: f5aae779491f41ada7162345278339d6, State: Running, Role: LEADER
I20260807 18:26:51.441052 27691 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260807 18:26:51.441802 27679 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:51.444408 27694 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5aae779491f41ada7162345278339d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260807 18:26:51.445106 27694 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:51.445322 27696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5aae779491f41ada7162345278339d6. Latest consensus state: current_term: 1 leader_uuid: "f5aae779491f41ada7162345278339d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aae779491f41ada7162345278339d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260807 18:26:51.445652 27696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:51.447343 27706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:51.452593 27706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:51.462797 27706 catalog_manager.cc:1347] Generated new cluster ID: dd722838996d4979a90651bcbe9f4a80
I20260807 18:26:51.462939 27706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:51.475438 27706 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:51.475950 27706 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:51.485183 27706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5aae779491f41ada7162345278339d6: Generated new TSK 0
I20260807 18:26:51.486080 27706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:51.500571 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27589
W20260807 18:26:51.501893 27672 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44043
W20260807 18:26:51.502660 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27589: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:51.502807 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27589
2026-08-07T18:26:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:51 dist-test-slave-sb2z krb5kdc[27728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:51 dist-test-slave-sb2z krb5kdc[27728](info): set up 2 sockets
Aug 07 18:26:51 dist-test-slave-sb2z krb5kdc[27728](info): commencing operation
krb5kdc: starting...
W20260807 18:26:52.846690 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:53 dist-test-slave-sb2z krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:53.216276 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:53.580987 27751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:53.581403 27751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:53.581506 27751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:53.589244 27751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:53.589389 27751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:53.589440 27751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:53.589504 27751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:53.589551 27751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:53.600387 27751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:53.601815 27751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:53.603154 27751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:53.616468 27760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:53.616671 27758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:53.616671 27757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:53.618029 27751 server_base.cc:1034] running on GCE node
I20260807 18:26:53.618880 27751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:53.620623 27751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:53.621883 27751 hybrid_clock.cc:648] HybridClock initialized: now 1786127213621834 us; error 43 us; skew 500 ppm
I20260807 18:26:53.622431 27751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:53 dist-test-slave-sb2z krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:53.626253 27751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:53.628324 27751 webserver.cc:460] Webserver started at http://127.0.0.1:35421/ using document root <none> and password file <none>
I20260807 18:26:53.629091 27751 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:53.629253 27751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:53.629590 27751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:53.632941 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be"
format_stamp: "Formatted at 2026-08-07 18:26:53 on dist-test-slave-sb2z"
server_key: "1ded489f0df44942a0f1f7220ad4ad7d"
server_key_iv: "007bd59e253a44a8d4813423b266d5b1"
server_key_version: "encryptionkey@0"
I20260807 18:26:53.633868 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be"
format_stamp: "Formatted at 2026-08-07 18:26:53 on dist-test-slave-sb2z"
server_key: "1ded489f0df44942a0f1f7220ad4ad7d"
server_key_iv: "007bd59e253a44a8d4813423b266d5b1"
server_key_version: "encryptionkey@0"
I20260807 18:26:53.640239 27751 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:26:53.644719 27769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:53.646462 27751 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:26:53.646716 27751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be"
format_stamp: "Formatted at 2026-08-07 18:26:53 on dist-test-slave-sb2z"
server_key: "1ded489f0df44942a0f1f7220ad4ad7d"
server_key_iv: "007bd59e253a44a8d4813423b266d5b1"
server_key_version: "encryptionkey@0"
I20260807 18:26:53.646960 27751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:53.667018 27751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:53.672416 27751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:53.672781 27751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:53.696120 27751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46429
I20260807 18:26:53.696146 27833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46429 every 8 connection(s)
I20260807 18:26:53.697715 27751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:53.701834 27835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:53.704941 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27751
I20260807 18:26:53.705156 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:53.705718 30323 external_mini_cluster.cc:1428] Setting key 37c762b527de63688adbdd0820fe8757
I20260807 18:26:53.715340 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be: Bootstrap starting.
I20260807 18:26:53.719604 27835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:53.721204 27835 log.cc:824] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:26:53 dist-test-slave-sb2z krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:26:53.724855 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be: No bootstrap required, opened a new log
I20260807 18:26:53.732734 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:53.708086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:45985 (local address 127.0.0.1:46429)
0807 18:26:53.708984 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:53.709004 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:53.709052 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:26:53.710812 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:53.710837 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:53.711299 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:53.711736 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:53.713776 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:53.715159 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:53.717059 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:53.717823 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:53.717971 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:53.723955 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:53.723966 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:53.723983 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:53.724029 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:53.727680 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:53.728770 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:53.728778 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:53.728784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:53.728974 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:53.729486 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:53.729489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:53.729491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:53.730133 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:53.730402 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:53.731028 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 18:26:53.731491 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":438,"threads_started":1}
I20260807 18:26:53.734552 27835 raft_consensus.cc:348] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 18:26:53.734956 27835 raft_consensus.cc:374] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:53.735088 27835 raft_consensus.cc:729] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db42e6aeff4b4e8c9b5497af9ce3c7be, State: Initialized, Role: FOLLOWER
I20260807 18:26:53.735940 27835 consensus_queue.cc:260] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 18:26:53.736271 27835 raft_consensus.cc:388] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:53.736464 27835 raft_consensus.cc:482] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:53.736656 27835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:53.737782 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:53.739825 27835 raft_consensus.cc:504] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 18:26:53.740517 27835 leader_election.cc:302] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db42e6aeff4b4e8c9b5497af9ce3c7be; no voters:
I20260807 18:26:53.741488 27835 leader_election.cc:288] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:53.742017 27845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:53.744278 27835 sys_catalog.cc:564] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:53.749468 27845 raft_consensus.cc:686] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [term 1 LEADER]: Becoming Leader. State: Replica: db42e6aeff4b4e8c9b5497af9ce3c7be, State: Running, Role: LEADER
I20260807 18:26:53.750504 27845 consensus_queue.cc:237] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 18:26:53.754555 27849 sys_catalog.cc:455] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader db42e6aeff4b4e8c9b5497af9ce3c7be. Latest consensus state: current_term: 1 leader_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260807 18:26:53.755003 27849 sys_catalog.cc:458] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:53.755823 27848 sys_catalog.cc:455] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db42e6aeff4b4e8c9b5497af9ce3c7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260807 18:26:53.756170 27848 sys_catalog.cc:458] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:53.764393 27876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:53.770466 27876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:53.776067 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:53.754463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:41123 (local address 127.0.0.1:46429)
0807 18:26:53.754651 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:53.754657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:53.755042 (+ 385us) server_negotiation.cc:407] Connection header received
0807 18:26:53.755204 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:53.755211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:53.755302 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:53.755409 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:53.758103 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:53.759739 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:53.763066 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:53.763994 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:53.764147 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:53.767721 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:53.767729 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:53.767737 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:26:53.767790 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:53.772003 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:53.773186 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:53.773192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:53.773195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:53.773354 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:26:53.774013 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:26:53.774019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:26:53.774022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:26:53.774516 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:53.774794 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:53.775445 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 18:26:53.775701 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:26:53.781077 27876 catalog_manager.cc:1347] Generated new cluster ID: f7a346d7f9bb4135b0d0494fc89adc97
I20260807 18:26:53.781215 27876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:53.800343 27876 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:53.801870 27876 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:53.808269 27876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db42e6aeff4b4e8c9b5497af9ce3c7be: Generated new TSK 0
I20260807 18:26:53.809327 27876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:53.821588 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27751
I20260807 18:26:53.824079 27825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:26:53.918725 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27751
W20260807 18:26:53.949468 27726 connection.cc:535] client connection to 127.0.0.1:46429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46429): Connection reset by peer (error 104)
2026-08-07T18:26:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:54 dist-test-slave-sb2z krb5kdc[27884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:54 dist-test-slave-sb2z krb5kdc[27884](info): set up 2 sockets
Aug 07 18:26:54 dist-test-slave-sb2z krb5kdc[27884](info): commencing operation
krb5kdc: starting...
W20260807 18:26:55.335827 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.002s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:55 dist-test-slave-sb2z krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:55.728329 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:56.095300 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:56.095667 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:56.095768 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:56.103375 27898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:56.103505 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:56.103638 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:56.103711 27898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:56.103754 27898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:56.114056 27898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:56.115449 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:56.116899 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:56.129455 27906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:56.129467 27905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:56.130144 27909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:56.131038 27898 server_base.cc:1034] running on GCE node
I20260807 18:26:56.131969 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:56.133468 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:56.134703 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786127216134643 us; error 53 us; skew 500 ppm
I20260807 18:26:56.135150 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:56 dist-test-slave-sb2z krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:56.138746 27898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:56.140597 27898 webserver.cc:460] Webserver started at http://127.0.0.1:33723/ using document root <none> and password file <none>
I20260807 18:26:56.141340 27898 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:56.141508 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:56.141841 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:56.145107 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "d8bffaa4581948ce9fafeb156c0018d5"
format_stamp: "Formatted at 2026-08-07 18:26:56 on dist-test-slave-sb2z"
server_key: "22a84dd67f7148a93d379fbdf6e74304"
server_key_iv: "a0f8b519ff883e931107870793d211c9"
server_key_version: "encryptionkey@0"
I20260807 18:26:56.145920 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "d8bffaa4581948ce9fafeb156c0018d5"
format_stamp: "Formatted at 2026-08-07 18:26:56 on dist-test-slave-sb2z"
server_key: "22a84dd67f7148a93d379fbdf6e74304"
server_key_iv: "a0f8b519ff883e931107870793d211c9"
server_key_version: "encryptionkey@0"
I20260807 18:26:56.151638 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:26:56.155411 27920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:56.157141 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:26:56.157366 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "d8bffaa4581948ce9fafeb156c0018d5"
format_stamp: "Formatted at 2026-08-07 18:26:56 on dist-test-slave-sb2z"
server_key: "22a84dd67f7148a93d379fbdf6e74304"
server_key_iv: "a0f8b519ff883e931107870793d211c9"
server_key_version: "encryptionkey@0"
I20260807 18:26:56.157603 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:56.174926 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:56.179776 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:56.180110 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:56.201058 27898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39935
I20260807 18:26:56.201067 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39935 every 8 connection(s)
I20260807 18:26:56.202550 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:56.206459 27999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:56.209983 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 27898
I20260807 18:26:56.210206 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:56.210706 30323 external_mini_cluster.cc:1428] Setting key 088267fc555b6283171db597dccd692e
I20260807 18:26:56.220814 27999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5: Bootstrap starting.
I20260807 18:26:56.225059 27999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:56.226485 27999 log.cc:824] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:56.227113 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:56.212893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:33199 (local address 127.0.0.1:39935)
0807 18:26:56.213695 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:56.213709 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:56.213762 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:26:56.215845 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:56.215877 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:56.216469 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:56.216928 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:56.217492 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:56.219543 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:56.221462 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:56.222482 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:56.222704 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:56.224032 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:56.224047 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:56.224067 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:56.224811 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:56.224921 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:56.224961 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:56.225155 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:56.225494 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:26:56.225700 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":323,"threads_started":1}
I20260807 18:26:56.229840 27999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5: No bootstrap required, opened a new log
I20260807 18:26:56.231364 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:56.238751 27999 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260807 18:26:56.239255 27999 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:56.239375 27999 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bffaa4581948ce9fafeb156c0018d5, State: Initialized, Role: FOLLOWER
I20260807 18:26:56.240036 27999 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260807 18:26:56.240281 27999 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:56.240446 27999 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:56.240622 27999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:56.243839 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:56.236207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:53209 (local address 127.0.0.1:39935)
0807 18:26:56.236411 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:56.236417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:56.236784 (+ 367us) server_negotiation.cc:407] Connection header received
0807 18:26:56.236944 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:56.236950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:56.237055 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:56.237167 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:56.237705 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:56.238803 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:56.240752 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:26:56.241459 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:26:56.241600 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:26:56.242792 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:56.242798 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:56.242803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:56.242825 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:56.242891 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:56.242911 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:56.243027 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:56.243331 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:26:56.243387 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:26:56.243784 27999 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260807 18:26:56.244369 27999 leader_election.cc:302] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bffaa4581948ce9fafeb156c0018d5; no voters:
I20260807 18:26:56.245039 27999 leader_election.cc:288] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:56.245256 28015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:56.246527 28015 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [term 1 LEADER]: Becoming Leader. State: Replica: d8bffaa4581948ce9fafeb156c0018d5, State: Running, Role: LEADER
I20260807 18:26:56.247138 28015 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260807 18:26:56.247413 27999 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:56.250937 28018 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bffaa4581948ce9fafeb156c0018d5. Latest consensus state: current_term: 1 leader_uuid: "d8bffaa4581948ce9fafeb156c0018d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } } }
I20260807 18:26:56.251354 28018 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:56.251765 28017 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bffaa4581948ce9fafeb156c0018d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bffaa4581948ce9fafeb156c0018d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } } }
I20260807 18:26:56.252122 28017 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:56.253422 28031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:56.259366 28031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:56.269721 28031 catalog_manager.cc:1347] Generated new cluster ID: 5cca6899bfe84d29a47ebac529d9d741
I20260807 18:26:56.269874 28031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:56.294168 28031 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:56.295657 28031 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:56.303886 28031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8bffaa4581948ce9fafeb156c0018d5: Generated new TSK 0
I20260807 18:26:56.305055 28031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:56.319478 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27898
W20260807 18:26:56.320818 27994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:33199
W20260807 18:26:56.321856 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 27898: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:56.322104 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 27898
W20260807 18:26:56.350419 27882 connection.cc:535] client connection to 127.0.0.1:39935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39935): Connection reset by peer (error 104)
2026-08-07T18:26:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:56 dist-test-slave-sb2z krb5kdc[28045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:56 dist-test-slave-sb2z krb5kdc[28045](info): set up 2 sockets
Aug 07 18:26:56 dist-test-slave-sb2z krb5kdc[28045](info): commencing operation
krb5kdc: starting...
W20260807 18:26:57.710175 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:26:57 dist-test-slave-sb2z krb5kdc[28045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:26:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:26:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:26:58.084218 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:26:58.445233 28061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:26:58.445554 28061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:26:58.445721 28061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:26:58.453186 28061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:26:58.453331 28061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:26:58.453438 28061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:26:58.453557 28061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:26:58.453629 28061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:26:58.464100 28061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:26:58.465533 28061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:26:58.467007 28061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:26:58.479751 28067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:58.479770 28070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:26:58.480443 28066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:26:58.480870 28061 server_base.cc:1034] running on GCE node
I20260807 18:26:58.481784 28061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:26:58.483307 28061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:26:58.484576 28061 hybrid_clock.cc:648] HybridClock initialized: now 1786127218484516 us; error 62 us; skew 500 ppm
I20260807 18:26:58.484977 28061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:26:58 dist-test-slave-sb2z krb5kdc[28045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:26:58.488519 28061 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:26:58.490161 28061 webserver.cc:460] Webserver started at http://127.0.0.1:37881/ using document root <none> and password file <none>
I20260807 18:26:58.490962 28061 fs_manager.cc:359] Metadata directory not provided
I20260807 18:26:58.491127 28061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:26:58.491452 28061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:26:58.494642 28061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "1841abffc1de46088e6e1539452d99b3"
format_stamp: "Formatted at 2026-08-07 18:26:58 on dist-test-slave-sb2z"
server_key: "95e594c48c354ad196eb3a12e83647cb"
server_key_iv: "9f31858198a8c77ecf7f71dc2a584741"
server_key_version: "encryptionkey@0"
I20260807 18:26:58.495452 28061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "1841abffc1de46088e6e1539452d99b3"
format_stamp: "Formatted at 2026-08-07 18:26:58 on dist-test-slave-sb2z"
server_key: "95e594c48c354ad196eb3a12e83647cb"
server_key_iv: "9f31858198a8c77ecf7f71dc2a584741"
server_key_version: "encryptionkey@0"
I20260807 18:26:58.501139 28061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:26:58.505014 28079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:26:58.506494 28061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:26:58.506726 28061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "1841abffc1de46088e6e1539452d99b3"
format_stamp: "Formatted at 2026-08-07 18:26:58 on dist-test-slave-sb2z"
server_key: "95e594c48c354ad196eb3a12e83647cb"
server_key_iv: "9f31858198a8c77ecf7f71dc2a584741"
server_key_version: "encryptionkey@0"
I20260807 18:26:58.506978 28061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:26:58.513203 28061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:26:58.518204 28061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:26:58.518548 28061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:26:58.537621 28061 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41017
I20260807 18:26:58.537660 28148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41017 every 8 connection(s)
I20260807 18:26:58.539145 28061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:26:58.542254 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28061
I20260807 18:26:58.542467 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:26:58.543046 30323 external_mini_cluster.cc:1428] Setting key bfcfbeeea61f60fbbcc11038c21c6de1
I20260807 18:26:58.544154 28149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:26:58.551379 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:58.545093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:44095 (local address 127.0.0.1:41017)
0807 18:26:58.545934 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:58.545946 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:58.545991 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:26:58.547709 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:58.547733 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:58.548107 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:58.548533 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:58.549652 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:58.549658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:58.549671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:58.549686 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:58.549763 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:58.549783 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:58.550055 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:58.550208 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 18:26:58.550402 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":378,"threads_started":1}
I20260807 18:26:58.556021 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:26:58.558138 28149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3: Bootstrap starting.
I20260807 18:26:58.562189 28149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3: Neither blocks nor log segments found. Creating new log.
I20260807 18:26:58.563731 28149 log.cc:824] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:26:58.564055 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:26:58.560998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:49777 (local address 127.0.0.1:41017)
0807 18:26:58.561233 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:26:58.561241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:26:58.561609 (+ 368us) server_negotiation.cc:407] Connection header received
0807 18:26:58.561822 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:26:58.561826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:26:58.561879 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:26:58.561955 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:26:58.563184 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:26:58.563189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:26:58.563192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:26:58.563208 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:26:58.563250 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:26:58.563265 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:26:58.563369 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:26:58.563595 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:26:58.563656 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:26:58.567119 28149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3: No bootstrap required, opened a new log
I20260807 18:26:58.575313 28149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } }
I20260807 18:26:58.575882 28149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:26:58.576046 28149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1841abffc1de46088e6e1539452d99b3, State: Initialized, Role: FOLLOWER
I20260807 18:26:58.576658 28149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } }
I20260807 18:26:58.576925 28149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:26:58.577085 28149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:26:58.577286 28149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:26:58.579628 28149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } }
I20260807 18:26:58.580228 28149 leader_election.cc:302] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1841abffc1de46088e6e1539452d99b3; no voters:
I20260807 18:26:58.580845 28149 leader_election.cc:288] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:26:58.580968 28159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:26:58.581840 28159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [term 1 LEADER]: Becoming Leader. State: Replica: 1841abffc1de46088e6e1539452d99b3, State: Running, Role: LEADER
I20260807 18:26:58.582414 28159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } }
I20260807 18:26:58.583145 28149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:26:58.585631 28161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1841abffc1de46088e6e1539452d99b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } } }
I20260807 18:26:58.585537 28162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1841abffc1de46088e6e1539452d99b3. Latest consensus state: current_term: 1 leader_uuid: "1841abffc1de46088e6e1539452d99b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1841abffc1de46088e6e1539452d99b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41017 } } }
I20260807 18:26:58.586076 28161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:58.586112 28162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:26:58.588539 28168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:26:58.594014 28168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:26:58.603389 28168 catalog_manager.cc:1347] Generated new cluster ID: 8eb7b9ccec7e469a9dffb6666bc46f98
I20260807 18:26:58.603549 28168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:26:58.625468 28168 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:26:58.625993 28168 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:26:58.638794 28168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1841abffc1de46088e6e1539452d99b3: Generated new TSK 0
I20260807 18:26:58.639787 28168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:26:58.644865 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28061
W20260807 18:26:58.646195 28142 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:44095
W20260807 18:26:58.646900 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28061: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:26:58.647068 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28061
2026-08-07T18:26:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:26:58 dist-test-slave-sb2z krb5kdc[28188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:26:58 dist-test-slave-sb2z krb5kdc[28188](info): set up 2 sockets
Aug 07 18:26:58 dist-test-slave-sb2z krb5kdc[28188](info): commencing operation
krb5kdc: starting...
W20260807 18:27:00.000622 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:00 dist-test-slave-sb2z krb5kdc[28188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:27:00.375865 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:00.724908 28213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:00.725299 28213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:00.725481 28213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:00.732928 28213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:00.733106 28213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:00.733222 28213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:00.733338 28213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:00.733428 28213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:00.743599 28213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:00.745070 28213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:00.746487 28213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:00.759044 28226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:00.759169 28224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:00.759174 28228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:00.760298 28213 server_base.cc:1034] running on GCE node
I20260807 18:27:00.761234 28213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:00.762831 28213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:00.764117 28213 hybrid_clock.cc:648] HybridClock initialized: now 1786127220764059 us; error 54 us; skew 500 ppm
I20260807 18:27:00.764531 28213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:00 dist-test-slave-sb2z krb5kdc[28188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127220, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:00.768177 28213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:27:00.770473 28213 webserver.cc:460] Webserver started at http://127.0.0.1:42309/ using document root <none> and password file <none>
I20260807 18:27:00.771219 28213 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:00.771325 28213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:00.771657 28213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:00.774693 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7"
format_stamp: "Formatted at 2026-08-07 18:27:00 on dist-test-slave-sb2z"
server_key: "0dfc0c50c1b034423f8f63da15c92fd1"
server_key_iv: "c844913235317dff853f6f0b9e4b7f70"
server_key_version: "encryptionkey@0"
I20260807 18:27:00.775445 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7"
format_stamp: "Formatted at 2026-08-07 18:27:00 on dist-test-slave-sb2z"
server_key: "0dfc0c50c1b034423f8f63da15c92fd1"
server_key_iv: "c844913235317dff853f6f0b9e4b7f70"
server_key_version: "encryptionkey@0"
I20260807 18:27:00.781540 28213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:27:00.785425 28238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:00.787101 28213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:27:00.787335 28213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7"
format_stamp: "Formatted at 2026-08-07 18:27:00 on dist-test-slave-sb2z"
server_key: "0dfc0c50c1b034423f8f63da15c92fd1"
server_key_iv: "c844913235317dff853f6f0b9e4b7f70"
server_key_version: "encryptionkey@0"
I20260807 18:27:00.787598 28213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:00.812634 28213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:00.817627 28213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:00.817965 28213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:00.839260 28213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40441
I20260807 18:27:00.839305 28306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40441 every 8 connection(s)
I20260807 18:27:00.841037 28213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:00.842707 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28213
I20260807 18:27:00.842933 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:00.843549 30323 external_mini_cluster.cc:1428] Setting key 27d6267aeb9a1e6815a549f03fe305fb
I20260807 18:27:00.846557 28307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:27:00 dist-test-slave-sb2z krb5kdc[28188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:27:00.861295 28307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7: Bootstrap starting.
I20260807 18:27:00.866103 28307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:00.867643 28307 log.cc:824] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:00.871665 28307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7: No bootstrap required, opened a new log
I20260807 18:27:00.872645 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:00.845619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:60865 (local address 127.0.0.1:40441)
0807 18:27:00.846496 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:00.846515 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:00.846584 (+ 69us) server_negotiation.cc:407] Connection header received
0807 18:27:00.849242 (+ 2658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:00.849273 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:00.849852 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:00.850366 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:00.852014 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:00.854349 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:00.855940 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:00.857152 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:00.857371 (+ 219us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:00.862282 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:00.862295 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:00.862315 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:00.862367 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:00.866976 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:00.868087 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:00.868097 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:00.868106 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:00.868395 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:00.869112 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:00.869116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:00.869119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:00.869902 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:00.870249 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:00.870784 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:27:00.871140 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":370,"threads_started":1}
I20260807 18:27:00.877277 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:00.881280 28307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260807 18:27:00.881804 28307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:00.881945 28307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c9f104c68894e0b800b9fdf1d3d5ce7, State: Initialized, Role: FOLLOWER
I20260807 18:27:00.882689 28307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260807 18:27:00.882989 28307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:00.883152 28307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:00.883383 28307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:00.886569 28307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260807 18:27:00.887213 28307 leader_election.cc:302] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c9f104c68894e0b800b9fdf1d3d5ce7; no voters:
I20260807 18:27:00.887923 28307 leader_election.cc:288] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:00.888114 28323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:00.888864 28323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 3c9f104c68894e0b800b9fdf1d3d5ce7, State: Running, Role: LEADER
I20260807 18:27:00.889529 28323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260807 18:27:00.891463 28323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c9f104c68894e0b800b9fdf1d3d5ce7. Latest consensus state: current_term: 1 leader_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } } }
I20260807 18:27:00.891773 28323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:00.893093 28325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c9f104c68894e0b800b9fdf1d3d5ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } } }
I20260807 18:27:00.893424 28325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:00.897764 28307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:00.899114 28327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:00.902132 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:00.881968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:59927 (local address 127.0.0.1:40441)
0807 18:27:00.882168 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:00.882174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:00.883093 (+ 919us) server_negotiation.cc:407] Connection header received
0807 18:27:00.883359 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:00.883365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:00.883453 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:00.883618 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:00.885524 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:00.886368 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:00.888372 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:00.889200 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:00.889341 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:00.893665 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:00.893674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:00.893682 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:00.893738 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:00.898219 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:00.899341 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:00.899353 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:00.899359 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:00.899647 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:00.900210 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:00.900217 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:00.900221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:00.900740 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:00.901085 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:00.901490 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 18:27:00.901683 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:27:00.907085 28327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:00.917523 28327 catalog_manager.cc:1347] Generated new cluster ID: a94a12a028e345d0b066cd9e4e5fafe1
I20260807 18:27:00.917654 28327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:00.938696 28327 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:00.940305 28327 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:00.949337 28327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c9f104c68894e0b800b9fdf1d3d5ce7: Generated new TSK 0
I20260807 18:27:00.950219 28327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:00.959164 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28213
I20260807 18:27:00.960641 28301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:01.053138 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28213
W20260807 18:27:01.079903 28185 connection.cc:535] client connection to 127.0.0.1:40441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40441): Connection reset by peer (error 104)
2026-08-07T18:27:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:01 dist-test-slave-sb2z krb5kdc[28354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:01 dist-test-slave-sb2z krb5kdc[28354](info): set up 2 sockets
Aug 07 18:27:01 dist-test-slave-sb2z krb5kdc[28354](info): commencing operation
krb5kdc: starting...
W20260807 18:27:02.462963 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.004s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:02 dist-test-slave-sb2z krb5kdc[28354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:27:02.835449 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:03.190683 28373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:03.191012 28373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:03.191215 28373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:03.198664 28373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:03.198791 28373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:03.198879 28373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:03.198977 28373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:03.199079 28373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:03.209342 28373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:03.210794 28373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:03.212232 28373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:03.224931 28379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:03.225090 28380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:03.224929 28382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:03.226300 28373 server_base.cc:1034] running on GCE node
I20260807 18:27:03.227180 28373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:03.228775 28373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:03.229995 28373 hybrid_clock.cc:648] HybridClock initialized: now 1786127223229932 us; error 59 us; skew 500 ppm
I20260807 18:27:03.230409 28373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:03 dist-test-slave-sb2z krb5kdc[28354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127223, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:03.237951 28373 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:27:03.239841 28373 webserver.cc:460] Webserver started at http://127.0.0.1:35275/ using document root <none> and password file <none>
I20260807 18:27:03.240598 28373 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:03.240759 28373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:03.241129 28373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:03.244366 28373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "9681ad0d98ca4fe19a76a3dd948cd940"
format_stamp: "Formatted at 2026-08-07 18:27:03 on dist-test-slave-sb2z"
server_key: "9661c02fe4756e418a36a186cbbc327c"
server_key_iv: "709ae4ab2955f46832b418cc05c9211f"
server_key_version: "encryptionkey@0"
I20260807 18:27:03.245208 28373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "9681ad0d98ca4fe19a76a3dd948cd940"
format_stamp: "Formatted at 2026-08-07 18:27:03 on dist-test-slave-sb2z"
server_key: "9661c02fe4756e418a36a186cbbc327c"
server_key_iv: "709ae4ab2955f46832b418cc05c9211f"
server_key_version: "encryptionkey@0"
I20260807 18:27:03.250900 28373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:27:03.254818 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:03.256697 28373 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:27:03.256923 28373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "9681ad0d98ca4fe19a76a3dd948cd940"
format_stamp: "Formatted at 2026-08-07 18:27:03 on dist-test-slave-sb2z"
server_key: "9661c02fe4756e418a36a186cbbc327c"
server_key_iv: "709ae4ab2955f46832b418cc05c9211f"
server_key_version: "encryptionkey@0"
I20260807 18:27:03.257157 28373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:03.270062 28373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:03.274505 28373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:03.274813 28373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:03.295874 28373 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41761
I20260807 18:27:03.295881 28470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41761 every 8 connection(s)
I20260807 18:27:03.297601 28373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:03.301745 28472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:03.302639 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28373
I20260807 18:27:03.302840 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:03.303493 30323 external_mini_cluster.cc:1428] Setting key bc4bea05ce5f446ba01c8bace1961856
I20260807 18:27:03.316444 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940: Bootstrap starting.
I20260807 18:27:03.318960 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:03.305446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:60639 (local address 127.0.0.1:41761)
0807 18:27:03.306291 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:03.306302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:03.306349 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:27:03.308571 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:03.308600 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:03.309053 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:03.309409 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:03.310122 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:03.311431 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:03.313400 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:03.314182 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:03.314313 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:03.315771 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:03.315778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:03.315792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:03.316386 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:03.316472 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:03.316528 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:03.316704 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:03.317204 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:27:03.317480 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":373,"threads_started":1}
I20260807 18:27:03.321362 28472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:03.322772 28472 log.cc:824] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:03.325323 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:03.326889 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940: No bootstrap required, opened a new log
I20260807 18:27:03.336994 28472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260807 18:27:03.337432 28472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:03.337560 28472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9681ad0d98ca4fe19a76a3dd948cd940, State: Initialized, Role: FOLLOWER
I20260807 18:27:03.338312 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:03.330890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:37321 (local address 127.0.0.1:41761)
0807 18:27:03.331118 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:03.331125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:03.331585 (+ 460us) server_negotiation.cc:407] Connection header received
0807 18:27:03.331784 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:03.331790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:03.331868 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:03.331983 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:03.332572 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:03.333774 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:03.335189 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:03.335940 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:03.336089 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:03.337441 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:03.337450 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:03.337458 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:03.337481 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:03.337534 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:03.337564 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:03.337691 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:03.337928 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:27:03.337996 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:27:03.338440 28472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260807 18:27:03.338737 28472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:03.338896 28472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:03.339114 28472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:03.341663 28472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260807 18:27:03.342120 28472 leader_election.cc:302] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9681ad0d98ca4fe19a76a3dd948cd940; no voters:
I20260807 18:27:03.342715 28472 leader_election.cc:288] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:03.342880 28486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:03.345058 28472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:03.347806 28486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [term 1 LEADER]: Becoming Leader. State: Replica: 9681ad0d98ca4fe19a76a3dd948cd940, State: Running, Role: LEADER
I20260807 18:27:03.348471 28486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260807 18:27:03.351444 28488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9681ad0d98ca4fe19a76a3dd948cd940. Latest consensus state: current_term: 1 leader_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } } }
I20260807 18:27:03.351876 28488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:03.352470 28487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681ad0d98ca4fe19a76a3dd948cd940" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } } }
I20260807 18:27:03.352841 28487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:03.353085 28505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:03.360697 28505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:03.373864 28505 catalog_manager.cc:1347] Generated new cluster ID: 89e40c9b5bef4d398f0ea70eb33f471a
I20260807 18:27:03.374011 28505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:03.386291 28505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:03.388298 28505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:03.402109 28505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9681ad0d98ca4fe19a76a3dd948cd940: Generated new TSK 0
I20260807 18:27:03.403271 28505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:03.419313 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28373
W20260807 18:27:03.420637 28465 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:60639
W20260807 18:27:03.421468 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28373: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:03.421600 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28373
W20260807 18:27:03.445185 28352 connection.cc:535] client connection to 127.0.0.1:41761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41761): Connection reset by peer (error 104)
2026-08-07T18:27:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:03 dist-test-slave-sb2z krb5kdc[28514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:03 dist-test-slave-sb2z krb5kdc[28514](info): set up 2 sockets
Aug 07 18:27:03 dist-test-slave-sb2z krb5kdc[28514](info): commencing operation
krb5kdc: starting...
W20260807 18:27:04.845827 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.380s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:05 dist-test-slave-sb2z krb5kdc[28514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:27:05.223443 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:05.581828 28534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:05.582155 28534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:05.582350 28534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:05.589725 28534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:05.589843 28534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:05.589937 28534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:05.590042 28534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:05.590124 28534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:05.600467 28534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:05.601873 28534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:05.603269 28534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:05.615726 28541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:05.615698 28544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:05.616035 28542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:05.617365 28534 server_base.cc:1034] running on GCE node
I20260807 18:27:05.618422 28534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:05.620002 28534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:05.621214 28534 hybrid_clock.cc:648] HybridClock initialized: now 1786127225621162 us; error 40 us; skew 500 ppm
I20260807 18:27:05.621591 28534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:05 dist-test-slave-sb2z krb5kdc[28514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:05.624868 28534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:27:05.626417 28534 webserver.cc:460] Webserver started at http://127.0.0.1:36733/ using document root <none> and password file <none>
I20260807 18:27:05.627135 28534 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:05.627291 28534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:05.627686 28534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:05.630856 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "16967b8a1ce14cd2a747006351ac5185"
format_stamp: "Formatted at 2026-08-07 18:27:05 on dist-test-slave-sb2z"
server_key: "c6c74075edc3bb8722e4c583c388ec28"
server_key_iv: "0d508ca558d51adddf096f8241e99522"
server_key_version: "encryptionkey@0"
I20260807 18:27:05.631755 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "16967b8a1ce14cd2a747006351ac5185"
format_stamp: "Formatted at 2026-08-07 18:27:05 on dist-test-slave-sb2z"
server_key: "c6c74075edc3bb8722e4c583c388ec28"
server_key_iv: "0d508ca558d51adddf096f8241e99522"
server_key_version: "encryptionkey@0"
I20260807 18:27:05.637472 28534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 18:27:05.641426 28554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:05.642900 28534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:27:05.643108 28534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "16967b8a1ce14cd2a747006351ac5185"
format_stamp: "Formatted at 2026-08-07 18:27:05 on dist-test-slave-sb2z"
server_key: "c6c74075edc3bb8722e4c583c388ec28"
server_key_iv: "0d508ca558d51adddf096f8241e99522"
server_key_version: "encryptionkey@0"
I20260807 18:27:05.643343 28534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:05.649338 28534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:05.653532 28534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:05.653856 28534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:05.672972 28534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42773
I20260807 18:27:05.673012 28629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42773 every 8 connection(s)
I20260807 18:27:05.674456 28534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:05.678217 28630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:05.679705 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28534
I20260807 18:27:05.679895 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:05.680363 30323 external_mini_cluster.cc:1428] Setting key eced6a5fc7e991ad08ceefa9e9a2c602
I20260807 18:27:05.688419 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:05.682214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:49405 (local address 127.0.0.1:42773)
0807 18:27:05.683037 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:05.683050 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:05.683097 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:27:05.684848 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:05.684875 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:05.685243 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:05.685577 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:05.686709 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:05.686716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:05.686729 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:05.686743 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:05.686803 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:05.686827 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:05.686996 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:05.687174 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 18:27:05.687414 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":389,"threads_started":1}
I20260807 18:27:05.692615 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:05.692833 28630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185: Bootstrap starting.
I20260807 18:27:05.696446 28630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:05.697933 28630 log.cc:824] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:05.701098 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:05.697720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:49069 (local address 127.0.0.1:42773)
0807 18:27:05.697966 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:05.697972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:05.698384 (+ 412us) server_negotiation.cc:407] Connection header received
0807 18:27:05.698679 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:05.698688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:05.698774 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:05.698914 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:05.700200 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:05.700205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:05.700209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:05.700226 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:05.700269 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:05.700286 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:05.700397 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:05.700603 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 18:27:05.700660 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:27:05.701841 28630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185: No bootstrap required, opened a new log
I20260807 18:27:05.710069 28630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 18:27:05.710505 28630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:05.710630 28630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16967b8a1ce14cd2a747006351ac5185, State: Initialized, Role: FOLLOWER
I20260807 18:27:05.711246 28630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 18:27:05.711520 28630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:05.711738 28630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:05.711934 28630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:05.714357 28630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 18:27:05.714908 28630 leader_election.cc:302] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16967b8a1ce14cd2a747006351ac5185; no voters:
I20260807 18:27:05.715644 28630 leader_election.cc:288] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:05.715796 28643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:05.716609 28643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [term 1 LEADER]: Becoming Leader. State: Replica: 16967b8a1ce14cd2a747006351ac5185, State: Running, Role: LEADER
I20260807 18:27:05.717226 28643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 18:27:05.718237 28630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:05.720391 28645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16967b8a1ce14cd2a747006351ac5185" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260807 18:27:05.720878 28645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:05.721419 28646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16967b8a1ce14cd2a747006351ac5185. Latest consensus state: current_term: 1 leader_uuid: "16967b8a1ce14cd2a747006351ac5185" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16967b8a1ce14cd2a747006351ac5185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260807 18:27:05.721741 28646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:05.723078 28650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:05.730369 28650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:05.741147 28650 catalog_manager.cc:1347] Generated new cluster ID: f61edd3e3c244ab2844102cdc90792c2
I20260807 18:27:05.741297 28650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:05.759177 28650 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:05.759738 28650 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:05.775370 28650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16967b8a1ce14cd2a747006351ac5185: Generated new TSK 0
I20260807 18:27:05.776521 28650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:05.789801 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28534
W20260807 18:27:05.791388 28625 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:49405
W20260807 18:27:05.792378 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28534: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:05.792558 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28534
2026-08-07T18:27:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:05 dist-test-slave-sb2z krb5kdc[28671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:05 dist-test-slave-sb2z krb5kdc[28671](info): set up 2 sockets
Aug 07 18:27:05 dist-test-slave-sb2z krb5kdc[28671](info): commencing operation
krb5kdc: starting...
W20260807 18:27:07.142536 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.004s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:07 dist-test-slave-sb2z krb5kdc[28671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:07.507010 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43715
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:43715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:07.861529 28687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:07.861840 28687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:07.862021 28687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:07.869415 28687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:07.869535 28687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:07.869635 28687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:07.869729 28687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:07.869802 28687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:07.880234 28687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:43715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:43715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:07.881664 28687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:07.883049 28687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:07.895581 28695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:07.895797 28696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:07.896004 28698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:07.896857 28687 server_base.cc:1034] running on GCE node
I20260807 18:27:07.897678 28687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:07.899142 28687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:07.900380 28687 hybrid_clock.cc:648] HybridClock initialized: now 1786127227900313 us; error 65 us; skew 500 ppm
I20260807 18:27:07.900765 28687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:07 dist-test-slave-sb2z krb5kdc[28671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127227, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:07.904043 28687 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:07.905711 28687 webserver.cc:460] Webserver started at http://127.29.156.254:35337/ using document root <none> and password file <none>
I20260807 18:27:07.906385 28687 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:07.906495 28687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:07.906770 28687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:07.909873 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "630271e85e6b44e7855ecaf2941e11f4"
format_stamp: "Formatted at 2026-08-07 18:27:07 on dist-test-slave-sb2z"
server_key: "3537812fb5975d4171921df5ee90cc31"
server_key_iv: "05c8fcb57db46a5118e4e81816636a85"
server_key_version: "encryptionkey@0"
I20260807 18:27:07.910629 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "630271e85e6b44e7855ecaf2941e11f4"
format_stamp: "Formatted at 2026-08-07 18:27:07 on dist-test-slave-sb2z"
server_key: "3537812fb5975d4171921df5ee90cc31"
server_key_iv: "05c8fcb57db46a5118e4e81816636a85"
server_key_version: "encryptionkey@0"
I20260807 18:27:07.916002 28687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:27:07.919657 28706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:07.921132 28687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:27:07.921330 28687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "630271e85e6b44e7855ecaf2941e11f4"
format_stamp: "Formatted at 2026-08-07 18:27:07 on dist-test-slave-sb2z"
server_key: "3537812fb5975d4171921df5ee90cc31"
server_key_iv: "05c8fcb57db46a5118e4e81816636a85"
server_key_version: "encryptionkey@0"
I20260807 18:27:07.921547 28687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:07.938438 28687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:07.942973 28687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:07.943276 28687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:07.963608 28687 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:43715
I20260807 18:27:07.963703 28779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:43715 every 8 connection(s)
I20260807 18:27:07.965027 28687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:07.969110 28780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:07.972851 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28687
I20260807 18:27:07.973074 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:07.973711 30323 external_mini_cluster.cc:1428] Setting key 1f1dab059fbd776b5bb837dfc4bae61b
I20260807 18:27:07.982378 28780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4: Bootstrap starting.
I20260807 18:27:07.985842 28780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:07.986974 28780 log.cc:824] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:27:07 dist-test-slave-sb2z krb5kdc[28671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:07.990442 28780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4: No bootstrap required, opened a new log
I20260807 18:27:07.998185 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:07.975802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56405 (local address 127.29.156.254:43715)
0807 18:27:07.976541 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:07.976553 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:07.976602 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:27:07.978323 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:07.978346 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:07.978753 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:07.979150 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:07.980801 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:07.982723 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:07.984120 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:07.984788 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:07.984995 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:07.990067 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:07.990075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:07.990090 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:07.990124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:07.993433 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:07.994467 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:07.994473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:07.994479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:07.994650 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:07.995155 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:07.995159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:07.995161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:07.995966 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:07.996256 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:07.996756 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:27:07.997012 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":311,"threads_started":1}
I20260807 18:27:08.000720 28780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } }
I20260807 18:27:08.001250 28780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:08.001412 28780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630271e85e6b44e7855ecaf2941e11f4, State: Initialized, Role: FOLLOWER
I20260807 18:27:08.002012 28780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } }
I20260807 18:27:08.002300 28780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:08.002339 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:08.002468 28780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:08.002643 28780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:08.006364 28780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } }
I20260807 18:27:08.007242 28780 leader_election.cc:302] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 630271e85e6b44e7855ecaf2941e11f4; no voters:
I20260807 18:27:08.008075 28780 leader_election.cc:288] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:08.008206 28791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:08.010497 28780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:08.012939 28791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [term 1 LEADER]: Becoming Leader. State: Replica: 630271e85e6b44e7855ecaf2941e11f4, State: Running, Role: LEADER
I20260807 18:27:08.013639 28791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } }
I20260807 18:27:08.017066 28793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "630271e85e6b44e7855ecaf2941e11f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } } }
I20260807 18:27:08.017598 28793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:08.018301 28796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 630271e85e6b44e7855ecaf2941e11f4. Latest consensus state: current_term: 1 leader_uuid: "630271e85e6b44e7855ecaf2941e11f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630271e85e6b44e7855ecaf2941e11f4" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 43715 } } }
I20260807 18:27:08.018623 28807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:08.018730 28796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:08.035591 28807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:08.035794 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:08.009249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59441 (local address 127.29.156.254:43715)
0807 18:27:08.009422 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:08.009438 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:08.012385 (+ 2947us) server_negotiation.cc:407] Connection header received
0807 18:27:08.012670 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:08.012680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:08.012803 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:08.012958 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:08.015209 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:08.016590 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:08.018291 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:08.019043 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:08.019160 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:08.023577 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:08.023589 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:08.023597 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:08.023656 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:08.028989 (+ 5333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:08.030002 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:08.030008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:08.030012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:08.031789 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:08.032410 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:08.032415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:08.032418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:08.033051 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:08.033336 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:08.034670 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0807 18:27:08.034856 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:27:08.047286 28807 catalog_manager.cc:1347] Generated new cluster ID: e8c31bb2a1cc4b8f94fbf2c901daf48a
I20260807 18:27:08.047429 28807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:08.065014 28807 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:08.066540 28807 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:08.087483 28807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 630271e85e6b44e7855ecaf2941e11f4: Generated new TSK 0
I20260807 18:27:08.088706 28807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:08.103240 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28687
I20260807 18:27:08.104669 28773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:08.201457 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28687
W20260807 18:27:08.228338 28669 connection.cc:535] client connection to 127.29.156.254:43715 recv error: Network error: failed to read from TLS socket (remote: 127.29.156.254:43715): Connection reset by peer (error 104)
2026-08-07T18:27:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:08 dist-test-slave-sb2z krb5kdc[28828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:08 dist-test-slave-sb2z krb5kdc[28828](info): set up 2 sockets
Aug 07 18:27:08 dist-test-slave-sb2z krb5kdc[28828](info): commencing operation
krb5kdc: starting...
W20260807 18:27:09.581805 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:09 dist-test-slave-sb2z krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:09.956244 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46687
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:10.314517 28849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:10.314891 28849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:10.315083 28849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:10.322559 28849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:10.322728 28849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:10.322849 28849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:10.322959 28849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:10.323052 28849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:10.333112 28849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:46687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:10.334458 28849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:10.335937 28849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:10.348138 28855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:10.348136 28859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:10.348217 28857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:10.349110 28849 server_base.cc:1034] running on GCE node
I20260807 18:27:10.349979 28849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:10.351496 28849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:10.352785 28849 hybrid_clock.cc:648] HybridClock initialized: now 1786127230352735 us; error 33 us; skew 500 ppm
I20260807 18:27:10.353154 28849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:10 dist-test-slave-sb2z krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127230, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:10.356438 28849 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:10.357920 28849 webserver.cc:460] Webserver started at http://127.29.156.254:35307/ using document root <none> and password file <none>
I20260807 18:27:10.358587 28849 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:10.358692 28849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:10.358953 28849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:10.362182 28849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "db5deeb76b984fb5bf00055213f54778"
format_stamp: "Formatted at 2026-08-07 18:27:10 on dist-test-slave-sb2z"
server_key: "a7e3f3e6cdfb0ff2db6cc4546cc156b2"
server_key_iv: "df1c36bb5efad2b7adb67a71dbad12eb"
server_key_version: "encryptionkey@0"
I20260807 18:27:10.362932 28849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "db5deeb76b984fb5bf00055213f54778"
format_stamp: "Formatted at 2026-08-07 18:27:10 on dist-test-slave-sb2z"
server_key: "a7e3f3e6cdfb0ff2db6cc4546cc156b2"
server_key_iv: "df1c36bb5efad2b7adb67a71dbad12eb"
server_key_version: "encryptionkey@0"
I20260807 18:27:10.368381 28849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 18:27:10.372027 28866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:10.373420 28849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:27:10.373637 28849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "db5deeb76b984fb5bf00055213f54778"
format_stamp: "Formatted at 2026-08-07 18:27:10 on dist-test-slave-sb2z"
server_key: "a7e3f3e6cdfb0ff2db6cc4546cc156b2"
server_key_iv: "df1c36bb5efad2b7adb67a71dbad12eb"
server_key_version: "encryptionkey@0"
I20260807 18:27:10.373855 28849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:10.397099 28849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:10.401937 28849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:10.402283 28849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:10.423142 28849 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:46687
I20260807 18:27:10.423182 28936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:46687 every 8 connection(s)
I20260807 18:27:10.424628 28849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:10.428529 28937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:10.432654 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 28849
I20260807 18:27:10.432876 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:10.433367 30323 external_mini_cluster.cc:1428] Setting key 8dc9d9cce7d125d8f146ee7e46eb7c98
I20260807 18:27:10.441207 28937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778: Bootstrap starting.
I20260807 18:27:10.444854 28937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:10.446218 28937 log.cc:824] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:10.449872 28937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778: No bootstrap required, opened a new log
I20260807 18:27:10.449997 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:10.435342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50925 (local address 127.29.156.254:46687)
0807 18:27:10.436241 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:10.436255 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:10.436306 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:27:10.439281 (+ 2975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:10.439310 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:10.439872 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:10.440330 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:10.440858 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:10.442879 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:10.444529 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:10.445629 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:10.445817 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:10.447177 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:10.447185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:10.447198 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:10.447910 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:10.448007 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:10.448061 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:10.448217 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:10.448499 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:27:10.448668 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":389,"threads_started":1}
I20260807 18:27:10.454210 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:10.458393 28937 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } }
I20260807 18:27:10.458796 28937 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:10.458927 28937 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5deeb76b984fb5bf00055213f54778, State: Initialized, Role: FOLLOWER
I20260807 18:27:10.459643 28937 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } }
I20260807 18:27:10.459964 28937 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:10.460127 28937 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:10.460325 28937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:10.462675 28937 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } }
I20260807 18:27:10.463181 28937 leader_election.cc:302] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5deeb76b984fb5bf00055213f54778; no voters:
I20260807 18:27:10.463971 28937 leader_election.cc:288] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:10.464118 28951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:10.465256 28951 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [term 1 LEADER]: Becoming Leader. State: Replica: db5deeb76b984fb5bf00055213f54778, State: Running, Role: LEADER
I20260807 18:27:10.465879 28951 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } }
I20260807 18:27:10.466840 28937 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:10.466848 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:10.458885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38589 (local address 127.29.156.254:46687)
0807 18:27:10.459095 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:10.459105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:10.459697 (+ 592us) server_negotiation.cc:407] Connection header received
0807 18:27:10.459889 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:10.459892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:10.460003 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:10.460164 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:10.460846 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:10.461753 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:10.463375 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:10.464021 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:10.464216 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:10.465850 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:10.465856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:10.465861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:10.465886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:10.465969 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:10.465987 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:10.466150 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:10.466394 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 18:27:10.466455 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:27:10.477746 28953 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5deeb76b984fb5bf00055213f54778. Latest consensus state: current_term: 1 leader_uuid: "db5deeb76b984fb5bf00055213f54778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } } }
I20260807 18:27:10.478268 28953 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:10.478847 28954 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5deeb76b984fb5bf00055213f54778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5deeb76b984fb5bf00055213f54778" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 46687 } } }
I20260807 18:27:10.479255 28954 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:10.484097 28972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:10.490236 28972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:10.500123 28972 catalog_manager.cc:1347] Generated new cluster ID: aa944dc72b6249ccbe4aefae9f788659
I20260807 18:27:10.500263 28972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:10.522847 28972 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:10.524827 28972 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:10.542356 28972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5deeb76b984fb5bf00055213f54778: Generated new TSK 0
I20260807 18:27:10.543227 28972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:10.558651 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28849
W20260807 18:27:10.560032 28932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50925
W20260807 18:27:10.560927 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 28849: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:10.561100 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 28849
W20260807 18:27:10.586220 28824 connection.cc:535] client connection to 127.29.156.254:46687 recv error: Network error: failed to read from TLS socket (remote: 127.29.156.254:46687): Connection reset by peer (error 104)
2026-08-07T18:27:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:10 dist-test-slave-sb2z krb5kdc[28987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:10 dist-test-slave-sb2z krb5kdc[28987](info): set up 2 sockets
Aug 07 18:27:10 dist-test-slave-sb2z krb5kdc[28987](info): commencing operation
krb5kdc: starting...
W20260807 18:27:11.994977 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.389s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:12 dist-test-slave-sb2z krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:12.366995 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41259
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:46041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:12.720847 29004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:12.721191 29004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:12.721356 29004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:12.728873 29004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:12.729007 29004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:12.729066 29004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:12.729195 29004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:12.729267 29004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:12.739645 29004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:46041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:12.741074 29004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:12.742480 29004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:12.755331 29011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:12.755295 29018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:12.755373 29015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:12.756699 29004 server_base.cc:1034] running on GCE node
I20260807 18:27:12.757654 29004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:12.759259 29004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:12.760511 29004 hybrid_clock.cc:648] HybridClock initialized: now 1786127232760434 us; error 64 us; skew 500 ppm
I20260807 18:27:12.760954 29004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:12 dist-test-slave-sb2z krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127232, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:12.764426 29004 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:12.766111 29004 webserver.cc:460] Webserver started at http://127.29.156.254:41843/ using document root <none> and password file <none>
I20260807 18:27:12.766793 29004 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:12.766901 29004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:12.767177 29004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:12.770407 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "6392586b7a99480eba7b60427b8a9d10"
format_stamp: "Formatted at 2026-08-07 18:27:12 on dist-test-slave-sb2z"
server_key: "0734f4a9de6500ba7b3a62292b4094bb"
server_key_iv: "5abb7fe44463fc3de5f9535e2ae4d5c7"
server_key_version: "encryptionkey@0"
I20260807 18:27:12.771191 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "6392586b7a99480eba7b60427b8a9d10"
format_stamp: "Formatted at 2026-08-07 18:27:12 on dist-test-slave-sb2z"
server_key: "0734f4a9de6500ba7b3a62292b4094bb"
server_key_iv: "5abb7fe44463fc3de5f9535e2ae4d5c7"
server_key_version: "encryptionkey@0"
I20260807 18:27:12.776911 29004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:27:12.780743 29027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:12.782262 29004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:27:12.782470 29004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "6392586b7a99480eba7b60427b8a9d10"
format_stamp: "Formatted at 2026-08-07 18:27:12 on dist-test-slave-sb2z"
server_key: "0734f4a9de6500ba7b3a62292b4094bb"
server_key_iv: "5abb7fe44463fc3de5f9535e2ae4d5c7"
server_key_version: "encryptionkey@0"
I20260807 18:27:12.782691 29004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:12.789012 29004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:12.794838 29004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:12.795249 29004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:12.816768 29004 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:41259
I20260807 18:27:12.816807 29098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:41259 every 8 connection(s)
I20260807 18:27:12.818305 29004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:12.822598 29099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:12.823447 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29004
I20260807 18:27:12.823685 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:12.824182 30323 external_mini_cluster.cc:1428] Setting key 2d1ede83f44f2a9051104803016abe91
I20260807 18:27:12.834311 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:12.826187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36393 (local address 127.29.156.254:41259)
0807 18:27:12.827049 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:12.827065 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:12.827126 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:27:12.830013 (+ 2887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:12.830046 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:12.830541 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:12.831072 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:12.832242 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:12.832250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:12.832267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:12.832285 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:12.832358 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:12.832391 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:12.832629 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:12.832760 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 18:27:12.832980 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":336,"threads_started":1}
I20260807 18:27:12.838245 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10: Bootstrap starting.
I20260807 18:27:12.838560 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:12.841559 29099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:12.842844 29099 log.cc:824] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:12.845888 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10: No bootstrap required, opened a new log
I20260807 18:27:12.846246 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:12.843627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48411 (local address 127.29.156.254:41259)
0807 18:27:12.843842 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:12.843851 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:12.844197 (+ 346us) server_negotiation.cc:407] Connection header received
0807 18:27:12.844418 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:12.844423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:12.844485 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:12.844591 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:12.845566 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:12.845572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:12.845577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:12.845594 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:12.845642 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:12.845656 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:12.845778 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:12.845910 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 18:27:12.845973 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:27:12.854525 29099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } }
I20260807 18:27:12.854954 29099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:12.855069 29099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6392586b7a99480eba7b60427b8a9d10, State: Initialized, Role: FOLLOWER
I20260807 18:27:12.855729 29099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } }
I20260807 18:27:12.855984 29099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:12.856135 29099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:12.856310 29099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:12.859073 29099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } }
I20260807 18:27:12.859647 29099 leader_election.cc:302] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6392586b7a99480eba7b60427b8a9d10; no voters:
I20260807 18:27:12.860287 29099 leader_election.cc:288] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:12.860495 29115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:12.861330 29115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [term 1 LEADER]: Becoming Leader. State: Replica: 6392586b7a99480eba7b60427b8a9d10, State: Running, Role: LEADER
I20260807 18:27:12.861918 29115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } }
I20260807 18:27:12.862668 29099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:12.864796 29119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6392586b7a99480eba7b60427b8a9d10. Latest consensus state: current_term: 1 leader_uuid: "6392586b7a99480eba7b60427b8a9d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } } }
I20260807 18:27:12.865229 29119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:12.866170 29123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:12.866026 29117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6392586b7a99480eba7b60427b8a9d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392586b7a99480eba7b60427b8a9d10" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41259 } } }
I20260807 18:27:12.867296 29117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:12.873514 29123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:12.883888 29123 catalog_manager.cc:1347] Generated new cluster ID: 8fefcf82b9154bd381489eb095a76253
I20260807 18:27:12.884034 29123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:12.908193 29123 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:12.908684 29123 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:12.917994 29123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6392586b7a99480eba7b60427b8a9d10: Generated new TSK 0
I20260807 18:27:12.919070 29123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:12.927369 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29004
W20260807 18:27:12.928725 29093 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36393
W20260807 18:27:12.929612 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29004: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:12.929764 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29004
2026-08-07T18:27:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:13 dist-test-slave-sb2z krb5kdc[29146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:13 dist-test-slave-sb2z krb5kdc[29146](info): set up 2 sockets
Aug 07 18:27:13 dist-test-slave-sb2z krb5kdc[29146](info): commencing operation
krb5kdc: starting...
W20260807 18:27:14.284816 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:14 dist-test-slave-sb2z krb5kdc[29146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:14.643850 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40265
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:45843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:40265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:14.999878 29161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:15.000216 29161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:15.000394 29161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:15.008363 29161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:15.008522 29161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:15.008665 29161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:15.008773 29161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:15.008844 29161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:15.019294 29161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:45843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:40265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:40265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:15.020860 29161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:15.022346 29161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:15.035933 29167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:15.036026 29172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:15.036026 29169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:15.038050 29161 server_base.cc:1034] running on GCE node
I20260807 18:27:15.038975 29161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:15.040622 29161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:15.041869 29161 hybrid_clock.cc:648] HybridClock initialized: now 1786127235041819 us; error 43 us; skew 500 ppm
I20260807 18:27:15.042367 29161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:15 dist-test-slave-sb2z krb5kdc[29146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:15.050599 29161 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:15.052523 29161 webserver.cc:460] Webserver started at http://127.29.156.254:38615/ using document root <none> and password file <none>
I20260807 18:27:15.053287 29161 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:15.053459 29161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:15.053812 29161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:15.057107 29161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "8662d97604f246208238136a2d28cc62"
format_stamp: "Formatted at 2026-08-07 18:27:15 on dist-test-slave-sb2z"
server_key: "25047811a4a517f759d27448d9367324"
server_key_iv: "a3b8b6e2ac10ebedfa7d29bf4bf6c2a5"
server_key_version: "encryptionkey@0"
I20260807 18:27:15.057926 29161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "8662d97604f246208238136a2d28cc62"
format_stamp: "Formatted at 2026-08-07 18:27:15 on dist-test-slave-sb2z"
server_key: "25047811a4a517f759d27448d9367324"
server_key_iv: "a3b8b6e2ac10ebedfa7d29bf4bf6c2a5"
server_key_version: "encryptionkey@0"
I20260807 18:27:15.063413 29161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:27:15.067246 29179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:15.068661 29161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:27:15.068866 29161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "8662d97604f246208238136a2d28cc62"
format_stamp: "Formatted at 2026-08-07 18:27:15 on dist-test-slave-sb2z"
server_key: "25047811a4a517f759d27448d9367324"
server_key_iv: "a3b8b6e2ac10ebedfa7d29bf4bf6c2a5"
server_key_version: "encryptionkey@0"
I20260807 18:27:15.069145 29161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:15.101071 29161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:15.105885 29161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:15.106225 29161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:15.126140 29161 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:40265
I20260807 18:27:15.126161 29260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:40265 every 8 connection(s)
I20260807 18:27:15.127664 29161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:15.131400 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29161
I20260807 18:27:15.131628 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:15.131657 29263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:15.132221 30323 external_mini_cluster.cc:1428] Setting key 0f2e523b8e8f3ddd73f85e62f31c590e
I20260807 18:27:15.146839 29263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62: Bootstrap starting.
Aug 07 18:27:15 dist-test-slave-sb2z krb5kdc[29146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:15.150566 29263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:15.151814 29263 log.cc:824] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:15.154765 29263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62: No bootstrap required, opened a new log
I20260807 18:27:15.156767 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:15.134092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49875 (local address 127.29.156.254:40265)
0807 18:27:15.134945 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:15.134957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:15.135002 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:27:15.136622 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:15.136645 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:15.137045 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:15.137390 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:15.139214 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:15.140580 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:15.142449 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:15.143381 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:15.143597 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:15.148912 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:15.148924 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:15.148944 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:15.148988 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:15.152471 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:15.153279 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:15.153286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:15.153292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:15.153475 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:15.153906 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:15.153917 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:15.153920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:15.154452 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:15.154808 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:15.155283 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 18:27:15.155760 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":416,"threads_started":1}
I20260807 18:27:15.160741 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:15.162832 29263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } }
I20260807 18:27:15.163208 29263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:15.163337 29263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8662d97604f246208238136a2d28cc62, State: Initialized, Role: FOLLOWER
I20260807 18:27:15.164017 29263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } }
I20260807 18:27:15.164309 29263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:15.164460 29263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:15.164655 29263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:15.167485 29263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } }
I20260807 18:27:15.168066 29263 leader_election.cc:302] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8662d97604f246208238136a2d28cc62; no voters:
I20260807 18:27:15.168788 29263 leader_election.cc:288] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:15.168895 29278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:15.171052 29263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:15.171845 29278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [term 1 LEADER]: Becoming Leader. State: Replica: 8662d97604f246208238136a2d28cc62, State: Running, Role: LEADER
I20260807 18:27:15.172624 29278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } }
I20260807 18:27:15.175812 29279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8662d97604f246208238136a2d28cc62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } } }
I20260807 18:27:15.176296 29279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:15.176819 29280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8662d97604f246208238136a2d28cc62. Latest consensus state: current_term: 1 leader_uuid: "8662d97604f246208238136a2d28cc62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8662d97604f246208238136a2d28cc62" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 40265 } } }
I20260807 18:27:15.177650 29280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:15.180016 29289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:15.188454 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:15.165563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60175 (local address 127.29.156.254:40265)
0807 18:27:15.165769 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:15.165783 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:15.166239 (+ 456us) server_negotiation.cc:407] Connection header received
0807 18:27:15.166390 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:15.166395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:15.166473 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:15.166579 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:15.168229 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:15.169463 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:15.171254 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:15.172116 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:15.172250 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:15.176258 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:15.176268 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:15.176275 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:15.176332 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:15.183247 (+ 6915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:15.184465 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:15.184474 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:15.184480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:15.184688 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:15.185295 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:15.185303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:15.185309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:15.185791 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:15.186107 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:15.187620 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0807 18:27:15.187919 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:27:15.193778 29289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:15.203601 29289 catalog_manager.cc:1347] Generated new cluster ID: 431208c59e9947ab97892fb438b37a7c
I20260807 18:27:15.203740 29289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:15.230275 29289 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:15.231652 29289 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:15.239161 29289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8662d97604f246208238136a2d28cc62: Generated new TSK 0
I20260807 18:27:15.240090 29289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:15.251729 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29161
I20260807 18:27:15.252885 29255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:15.351431 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29161
2026-08-07T18:27:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:15 dist-test-slave-sb2z krb5kdc[29306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:15 dist-test-slave-sb2z krb5kdc[29306](info): set up 2 sockets
Aug 07 18:27:15 dist-test-slave-sb2z krb5kdc[29306](info): commencing operation
krb5kdc: starting...
W20260807 18:27:16.699378 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.005s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:16 dist-test-slave-sb2z krb5kdc[29306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:17.079885 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41399
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:17.436474 29328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:17.436776 29328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:17.436933 29328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:17.444343 29328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:17.444474 29328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:17.444581 29328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:17.444682 29328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:17.444762 29328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:17.455272 29328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:17.456736 29328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:17.458144 29328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:17.470242 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:17.470360 29338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:17.470238 29336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:17.470737 29328 server_base.cc:1034] running on GCE node
I20260807 18:27:17.471956 29328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:17.473377 29328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:17.474581 29328 hybrid_clock.cc:648] HybridClock initialized: now 1786127237474521 us; error 49 us; skew 500 ppm
I20260807 18:27:17.474937 29328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:17 dist-test-slave-sb2z krb5kdc[29306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127237, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:17.478379 29328 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:17.479965 29328 webserver.cc:460] Webserver started at http://127.29.156.254:41587/ using document root <none> and password file <none>
I20260807 18:27:17.480664 29328 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:17.480799 29328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:17.481132 29328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:17.484278 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "5d404c159fa74b2c9398640b39b43a47"
format_stamp: "Formatted at 2026-08-07 18:27:17 on dist-test-slave-sb2z"
server_key: "07002ec9d907217c1d5822f209d01391"
server_key_iv: "fbcd7cd03a8407bad46350ea1caaa7a4"
server_key_version: "encryptionkey@0"
I20260807 18:27:17.485066 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "5d404c159fa74b2c9398640b39b43a47"
format_stamp: "Formatted at 2026-08-07 18:27:17 on dist-test-slave-sb2z"
server_key: "07002ec9d907217c1d5822f209d01391"
server_key_iv: "fbcd7cd03a8407bad46350ea1caaa7a4"
server_key_version: "encryptionkey@0"
I20260807 18:27:17.490518 29328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:27:17.494234 29350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:17.495878 29328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:27:17.496111 29328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "5d404c159fa74b2c9398640b39b43a47"
format_stamp: "Formatted at 2026-08-07 18:27:17 on dist-test-slave-sb2z"
server_key: "07002ec9d907217c1d5822f209d01391"
server_key_iv: "fbcd7cd03a8407bad46350ea1caaa7a4"
server_key_version: "encryptionkey@0"
I20260807 18:27:17.496294 29328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:17.522384 29328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:17.526970 29328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:17.527279 29328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:17.547371 29328 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:41399
I20260807 18:27:17.547402 29424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:41399 every 8 connection(s)
I20260807 18:27:17.549093 29328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:17.553004 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:17.557358 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29328
I20260807 18:27:17.557557 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:17.558106 30323 external_mini_cluster.cc:1428] Setting key 2d2a04e3f32d0b56377208d823fa39bb
I20260807 18:27:17.564824 29426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47: Bootstrap starting.
I20260807 18:27:17.568480 29426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:17.569803 29426 log.cc:824] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:17.572913 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:17.560051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40417 (local address 127.29.156.254:41399)
0807 18:27:17.560978 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:17.560990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:17.561040 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:27:17.563254 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:17.563277 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:17.563759 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:17.564076 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:17.564715 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:17.566083 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:17.568042 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:17.568777 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:17.568927 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:17.570200 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:17.570207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:17.570220 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:17.571075 (+ 855us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:17.571181 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:17.571219 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:17.571346 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:17.571663 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 18:27:17.571950 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":431,"threads_started":1}
I20260807 18:27:17.573107 29426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47: No bootstrap required, opened a new log
I20260807 18:27:17.576648 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:17.583043 29426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } }
I20260807 18:27:17.583614 29426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:17.583730 29426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d404c159fa74b2c9398640b39b43a47, State: Initialized, Role: FOLLOWER
I20260807 18:27:17.584331 29426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } }
I20260807 18:27:17.584587 29426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:17.584743 29426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:17.584951 29426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:17.588240 29426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } }
I20260807 18:27:17.588912 29426 leader_election.cc:302] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d404c159fa74b2c9398640b39b43a47; no voters:
I20260807 18:27:17.589267 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:17.581955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47565 (local address 127.29.156.254:41399)
0807 18:27:17.582157 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:17.582163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:17.582491 (+ 328us) server_negotiation.cc:407] Connection header received
0807 18:27:17.582705 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:17.582708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:17.582785 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:17.582908 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:17.583663 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:17.584765 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:17.586683 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:17.587158 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:17.587254 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:17.588436 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:17.588443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:17.588451 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:17.588473 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:17.588536 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:17.588553 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:17.588655 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:17.588794 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:27:17.588866 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:27:17.589845 29426 leader_election.cc:288] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:17.590014 29438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:17.592830 29426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:17.593348 29438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [term 1 LEADER]: Becoming Leader. State: Replica: 5d404c159fa74b2c9398640b39b43a47, State: Running, Role: LEADER
I20260807 18:27:17.594048 29438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } }
I20260807 18:27:17.597460 29439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d404c159fa74b2c9398640b39b43a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } } }
I20260807 18:27:17.597880 29439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:17.599152 29447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d404c159fa74b2c9398640b39b43a47. Latest consensus state: current_term: 1 leader_uuid: "5d404c159fa74b2c9398640b39b43a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d404c159fa74b2c9398640b39b43a47" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41399 } } }
I20260807 18:27:17.599483 29447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:17.599540 29452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:17.605329 29452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:17.616472 29452 catalog_manager.cc:1347] Generated new cluster ID: c44cc31a83e649d3aaa1afe049b3bb9d
I20260807 18:27:17.616605 29452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:17.660617 29452 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:17.662029 29452 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:17.671072 29452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d404c159fa74b2c9398640b39b43a47: Generated new TSK 0
I20260807 18:27:17.672024 29452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:17.683863 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29328
W20260807 18:27:17.685016 29419 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40417
W20260807 18:27:17.685782 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29328: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:17.685950 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29328
2026-08-07T18:27:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:17 dist-test-slave-sb2z krb5kdc[29472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:17 dist-test-slave-sb2z krb5kdc[29472](info): set up 2 sockets
Aug 07 18:27:17 dist-test-slave-sb2z krb5kdc[29472](info): commencing operation
krb5kdc: starting...
W20260807 18:27:19.050472 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.007s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:19 dist-test-slave-sb2z krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:19.423288 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41671
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:41671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:19.780215 29491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:19.780539 29491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:19.780704 29491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:19.788580 29491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:19.788719 29491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:19.788827 29491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:19.788920 29491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:19.789011 29491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:19.799784 29491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:41671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:41671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:19.801328 29491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:19.802875 29491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:19.815997 29504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:19.816198 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:19.816192 29499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:19.817196 29491 server_base.cc:1034] running on GCE node
I20260807 18:27:19.818464 29491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:19.820123 29491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:19.821400 29491 hybrid_clock.cc:648] HybridClock initialized: now 1786127239821351 us; error 43 us; skew 500 ppm
I20260807 18:27:19.821808 29491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:19 dist-test-slave-sb2z krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127239, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:19.825394 29491 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:19.827034 29491 webserver.cc:460] Webserver started at http://127.29.156.254:37855/ using document root <none> and password file <none>
I20260807 18:27:19.827829 29491 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:19.827942 29491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:19.828272 29491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:19.831770 29491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6"
format_stamp: "Formatted at 2026-08-07 18:27:19 on dist-test-slave-sb2z"
server_key: "691c96491ed404f222589356364f2b0f"
server_key_iv: "799680c674ee670c8c83ee94c74a7f4e"
server_key_version: "encryptionkey@0"
I20260807 18:27:19.832629 29491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6"
format_stamp: "Formatted at 2026-08-07 18:27:19 on dist-test-slave-sb2z"
server_key: "691c96491ed404f222589356364f2b0f"
server_key_iv: "799680c674ee670c8c83ee94c74a7f4e"
server_key_version: "encryptionkey@0"
I20260807 18:27:19.838497 29491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:27:19.842463 29513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:19.844501 29491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:27:19.844672 29491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6"
format_stamp: "Formatted at 2026-08-07 18:27:19 on dist-test-slave-sb2z"
server_key: "691c96491ed404f222589356364f2b0f"
server_key_iv: "799680c674ee670c8c83ee94c74a7f4e"
server_key_version: "encryptionkey@0"
I20260807 18:27:19.844867 29491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:19.852228 29491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:19.858762 29491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:19.859063 29491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:19.879904 29491 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:41671
I20260807 18:27:19.879959 29587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:41671 every 8 connection(s)
I20260807 18:27:19.881443 29491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:19.885664 29589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:19.890740 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29491
I20260807 18:27:19.890969 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:19.891563 30323 external_mini_cluster.cc:1428] Setting key 4336bc6334fe2ed80872b97c1c650125
I20260807 18:27:19.899115 29589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6: Bootstrap starting.
I20260807 18:27:19.900990 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:19.893836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38607 (local address 127.29.156.254:41671)
0807 18:27:19.894700 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:19.894712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:19.894763 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:27:19.896953 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:19.896981 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:19.897362 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:19.897723 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:19.899202 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:19.899209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:19.899223 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:19.899238 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:19.899350 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:19.899382 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:19.899641 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:19.899753 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 18:27:19.899949 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":377,"threads_started":1}
I20260807 18:27:19.903402 29589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:19.904743 29589 log.cc:824] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:19.905084 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:19.908452 29589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6: No bootstrap required, opened a new log
I20260807 18:27:19.913556 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:19.910717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40647 (local address 127.29.156.254:41671)
0807 18:27:19.910922 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:19.910928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:19.911372 (+ 444us) server_negotiation.cc:407] Connection header received
0807 18:27:19.911518 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:19.911544 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:19.911626 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:19.911749 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:19.912896 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:19.912903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:19.912907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:27:19.912926 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:19.912977 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:27:19.913004 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:19.913138 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:19.913188 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 18:27:19.913255 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:27:19.918359 29589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } }
I20260807 18:27:19.918870 29589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:27:19.919003 29589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b198b23e6104f2e97ac2dbfe15ba0d6, State: Initialized, Role: FOLLOWER
I20260807 18:27:19.919744 29589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } }
I20260807 18:27:19.919968 29589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:27:19.920074 29589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:27:19.920217 29589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:19.922520 29589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } }
I20260807 18:27:19.923043 29589 leader_election.cc:302] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b198b23e6104f2e97ac2dbfe15ba0d6; no voters:
I20260807 18:27:19.923822 29589 leader_election.cc:288] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:27:19.924010 29605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:19.925017 29605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6b198b23e6104f2e97ac2dbfe15ba0d6, State: Running, Role: LEADER
I20260807 18:27:19.925598 29605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } }
I20260807 18:27:19.926211 29589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:19.928845 29607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } } }
I20260807 18:27:19.929270 29607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:19.929584 29609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b198b23e6104f2e97ac2dbfe15ba0d6. Latest consensus state: current_term: 1 leader_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b198b23e6104f2e97ac2dbfe15ba0d6" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 41671 } } }
I20260807 18:27:19.929968 29609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:19.932157 29615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:19.937788 29615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:19.948565 29615 catalog_manager.cc:1347] Generated new cluster ID: e84e56112f834d85947c71deb40e73ae
I20260807 18:27:19.948737 29615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:19.997335 29615 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:19.997841 29615 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:20.013511 29615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b198b23e6104f2e97ac2dbfe15ba0d6: Generated new TSK 0
I20260807 18:27:20.014539 29615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:27:20.026394 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29491
W20260807 18:27:20.027673 29581 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38607
W20260807 18:27:20.028568 30323 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29491: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:27:20.028749 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29491
2026-08-07T18:27:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2343 ms)
[----------] 18 tests from AuthTokenIssuingTest (42564 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:20 dist-test-slave-sb2z krb5kdc[29636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:20 dist-test-slave-sb2z krb5kdc[29636](info): set up 2 sockets
Aug 07 18:27:20 dist-test-slave-sb2z krb5kdc[29636](info): commencing operation
krb5kdc: starting...
W20260807 18:27:21.386816 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.005s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:21 dist-test-slave-sb2z krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:21.764365 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36767
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:39187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:22.134063 29654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:22.134441 29654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:22.134527 29654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:22.134611 29654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:22.142129 29654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:22.142226 29654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:22.142284 29654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:22.142329 29654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:22.142396 29654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:22.152462 29654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:39187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:36767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:22.153784 29654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:22.155102 29654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:22.167639 29662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:22.167604 29665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:22.167691 29667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:22.168459 29654 server_base.cc:1034] running on GCE node
I20260807 18:27:22.169322 29654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:22.170770 29654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:22.172024 29654 hybrid_clock.cc:648] HybridClock initialized: now 1786127242171941 us; error 58 us; skew 500 ppm
I20260807 18:27:22.172519 29654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:22.177170 29654 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:22.178813 29654 webserver.cc:460] Webserver started at http://127.29.156.254:43817/ using document root <none> and password file <none>
I20260807 18:27:22.179621 29654 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:22.179747 29654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:22.180078 29654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:22.183230 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "9f35f07f30eb4cdfae32835a9fee5d6d"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "1ba0559234ebc6fcb58759a3ddafb84c"
server_key_iv: "7c67873a3569bdb1b586203deb03d6a0"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.184161 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "9f35f07f30eb4cdfae32835a9fee5d6d"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "1ba0559234ebc6fcb58759a3ddafb84c"
server_key_iv: "7c67873a3569bdb1b586203deb03d6a0"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.189495 29654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:27:22.193202 29675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:22.194645 29654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:27:22.194864 29654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "9f35f07f30eb4cdfae32835a9fee5d6d"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "1ba0559234ebc6fcb58759a3ddafb84c"
server_key_iv: "7c67873a3569bdb1b586203deb03d6a0"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.195125 29654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:22.229348 29654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:22.234227 29654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:22.234546 29654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:22.254534 29654 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:36767
I20260807 18:27:22.254557 29744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:36767 every 8 connection(s)
I20260807 18:27:22.255997 29654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:22.259958 29745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:22.264289 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29654
I20260807 18:27:22.264528 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:22.265383 30323 external_mini_cluster.cc:1428] Setting key 318a7fb81ec1ecd69fad7389f7859266
I20260807 18:27:22.268123 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:22.296072 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.272101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36767 (local address 127.0.0.1:60728)
0807 18:27:22.272990 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.273038 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:22.276462 (+ 3424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:22.278060 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:22.278067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:22.278786 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:22.279137 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.279156 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.280757 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.280766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.282519 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.282538 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.283580 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.283591 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.283849 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.284597 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:22.284632 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:22.287708 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:22.291336 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.291349 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.291363 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.291880 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.292333 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.292341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.292346 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.292519 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.293338 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:22.293352 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:22.293590 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 18:27:22.293859 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 18:27:22.294528 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":150,"thread_start_us":382,"threads_started":1}
I20260807 18:27:22.296123 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.273426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.29.156.254:36767)
0807 18:27:22.277411 (+ 3985us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:22.277417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:22.277448 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:27:22.277593 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:22.277598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:22.277816 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:22.278026 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:22.279354 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.280594 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.282701 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.283274 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.283452 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.287977 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:22.287990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:22.288003 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:22.288036 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:22.291166 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.292033 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.292044 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.292048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.292213 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.292641 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.292652 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.292655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.293189 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:22.293425 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:22.294015 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 18:27:22.294381 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3828,"thread_start_us":3695,"threads_started":1}
I20260807 18:27:22.300464 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.156.253@KRBTEST.COM; defaulting to no policy
I20260807 18:27:22.301621 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.301208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37862)
0807 18:27:22.301384 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.301503 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:22.301920 29676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
I20260807 18:27:22.303670 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.303201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37876)
0807 18:27:22.303370 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.303436 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:27:22.303923 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:43453: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
I20260807 18:27:22.347431 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:43453
--webserver_interface=127.29.156.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:39187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:27:22.353389 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } attempt: 1
I20260807 18:27:22.354760 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.354203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37880)
0807 18:27:22.354455 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.354610 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:27:22.355993 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.355589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37896)
0807 18:27:22.355753 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.355838 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:27:22.356230 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:43453: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
I20260807 18:27:22.424539 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } attempt: 2
I20260807 18:27:22.425964 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.425350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37912)
0807 18:27:22.425653 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.425762 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:27:22.427091 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.426694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37926)
0807 18:27:22.426880 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.426955 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:27:22.427310 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:43453: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
I20260807 18:27:22.518628 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } attempt: 3
I20260807 18:27:22.519907 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.519414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37930)
0807 18:27:22.519662 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.519762 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:27:22.520977 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.520601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37932)
0807 18:27:22.520750 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.520826 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:27:22.521186 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:43453: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
I20260807 18:27:22.664466 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } attempt: 4
I20260807 18:27:22.665716 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.665142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37938)
0807 18:27:22.665487 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.665577 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260807 18:27:22.666787 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.666403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37942)
0807 18:27:22.666584 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.666668 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:27:22.666970 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:43453: Network error: Client connection negotiation failed: client connection to 127.29.156.253:43453: connect: Connection refused (error 111)
W20260807 18:27:22.702924 29760 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:22.703344 29760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:22.703480 29760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:22.703642 29760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:22.711331 29760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:22.711452 29760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:22.711516 29760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:22.711652 29760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:22.711726 29760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:22.721925 29760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:39187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:43453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.156.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:22.723328 29760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:22.724781 29760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:22.738016 29765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:22.738073 29766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:22.738284 29768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:22.744386 29760 server_base.cc:1034] running on GCE node
I20260807 18:27:22.745406 29760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:22.747013 29760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:22.748303 29760 hybrid_clock.cc:648] HybridClock initialized: now 1786127242748226 us; error 50 us; skew 500 ppm
I20260807 18:27:22.748673 29760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:22.752228 29760 init.cc:377] Logged in from keytab as kudu/127.29.156.253@KRBTEST.COM (short username kudu)
I20260807 18:27:22.754114 29760 webserver.cc:460] Webserver started at http://127.29.156.253:35625/ using document root <none> and password file <none>
I20260807 18:27:22.754868 29760 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:22.754966 29760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:22.755263 29760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:22.758456 29760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data/instance:
uuid: "d3fbd8689cc3495093f8bd6c361b4424"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "a753125573c306de5c8c854ca6d8eda4"
server_key_iv: "87424e0b6c0fce4aa7b693c1695aa5ee"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.759251 29760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal/instance:
uuid: "d3fbd8689cc3495093f8bd6c361b4424"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "a753125573c306de5c8c854ca6d8eda4"
server_key_iv: "87424e0b6c0fce4aa7b693c1695aa5ee"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.764931 29760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:27:22.768666 29778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:22.770056 29760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:27:22.770290 29760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "d3fbd8689cc3495093f8bd6c361b4424"
format_stamp: "Formatted at 2026-08-07 18:27:22 on dist-test-slave-sb2z"
server_key: "a753125573c306de5c8c854ca6d8eda4"
server_key_iv: "87424e0b6c0fce4aa7b693c1695aa5ee"
server_key_version: "encryptionkey@0"
I20260807 18:27:22.770502 29760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:22.792192 29760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:22.797297 29760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:22.797608 29760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:22.819062 29760 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.253:43453
I20260807 18:27:22.819118 29857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.253:43453 every 8 connection(s)
I20260807 18:27:22.820608 29760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
I20260807 18:27:22.824571 29859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:22.824600 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29760
I20260807 18:27:22.824820 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-1/wal/instance
I20260807 18:27:22.825309 30323 external_mini_cluster.cc:1428] Setting key 8d79387f59e92cf476a6af668cf2c78e
I20260807 18:27:22.831631 29859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:22.856873 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.834407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.29.156.254:36767)
0807 18:27:22.835001 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:22.835011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:22.835976 (+ 965us) server_negotiation.cc:407] Connection header received
0807 18:27:22.837996 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:22.838004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:22.838107 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:22.838252 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:22.839907 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.841168 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.844196 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.844643 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.844756 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.850010 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:22.850015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:22.850018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:22.850072 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:22.852860 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.854014 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.854018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.854020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.854163 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.854733 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.854737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.854739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.855049 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:22.855188 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:22.856396 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0807 18:27:22.856589 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":288,"threads_started":1}
I20260807 18:27:22.858644 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.834951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36767 (local address 127.0.0.1:60730)
0807 18:27:22.835795 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.835843 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:22.837706 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:22.838455 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:22.838462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:22.838994 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:22.839680 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.839700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.841386 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.841396 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.843979 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.844000 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.844905 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.844916 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.845323 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.846186 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:22.846224 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:22.849726 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:22.853020 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.853033 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.853056 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.853863 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.854284 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.854292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.854299 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.854575 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.855202 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:22.855214 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:22.855789 (+ 575us) client_negotiation.cc:769] Sending connection context
0807 18:27:22.856266 (+ 477us) client_negotiation.cc:240] Negotiation successful
0807 18:27:22.856914 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":341,"threads_started":1}
I20260807 18:27:22.860800 29859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.156.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.252@KRBTEST.COM" created.
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:22.877429 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.861626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37950)
0807 18:27:22.861842 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.861865 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:22.862014 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:22.863071 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:22.863077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:22.863610 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:22.863851 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.863867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.865361 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.865365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.866349 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.866363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.867480 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.867487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.867775 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.868496 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:22.868525 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:22.871141 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:22.874775 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.874780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.874788 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.875135 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.875567 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.875571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.875584 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.875782 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.876392 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:22.876397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:22.876560 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 18:27:22.876703 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 18:27:22.877001 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:27:22.877537 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.861849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.29.156.253:43453)
0807 18:27:22.862609 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:22.862615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:22.862642 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:27:22.862688 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:22.862693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:22.862877 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:22.863001 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:22.864045 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.865227 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.866506 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.867305 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.867450 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.871356 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:22.871367 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:22.871382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:22.871417 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:22.874614 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.875272 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.875282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.875286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.875415 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.875891 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.875896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.875899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.876270 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:22.876497 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:22.876864 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 18:27:22.877136 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":603,"thread_start_us":439,"threads_started":1}
I20260807 18:27:22.880187 29859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:22.881487 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.881030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56336)
0807 18:27:22.881206 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.881336 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:27:22.881783 29782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:22.883497 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.883140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56348)
0807 18:27:22.883301 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.883380 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:27:22.883770 29859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
I20260807 18:27:22.911783 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:35557
--webserver_interface=127.29.156.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:39187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:27:22.933131 29859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 1
I20260807 18:27:22.934473 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.933841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56352)
0807 18:27:22.934116 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.934257 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:27:22.935859 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.935341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56358)
0807 18:27:22.935590 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.935697 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 18:27:22.936136 29859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:22.966212 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } attempt: 5
Aug 07 18:27:22 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:22.980463 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.966940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37962)
0807 18:27:22.967128 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.967151 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:22.967288 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:22.967713 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:22.967721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:22.968140 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:22.968329 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.968339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.969379 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.969384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.970727 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:22.970740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.971501 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.971508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:22.971764 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.972442 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:22.972459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:22.974788 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:22.977921 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.977926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.977930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.978321 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.978738 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:22.978743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:22.978747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:22.978886 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:22.979419 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:22.979423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:22.979627 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 18:27:22.979767 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:27:22.980087 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:27:22.980459 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.967061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.29.156.253:43453)
0807 18:27:22.967282 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:22.967288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:22.967311 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:27:22.967409 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:22.967414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:22.967499 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:22.967643 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:22.968485 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.969215 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.970878 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:22.971348 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:22.971485 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:22.974981 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:22.974986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:22.974989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:22.975023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:22.977781 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.978448 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.978458 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.978460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.978581 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:22.979000 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:22.979005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:22.979008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:22.979299 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:22.979452 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:22.979888 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 18:27:22.980111 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:27:22.981627 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:22.982930 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.982495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56372)
0807 18:27:22.982683 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.982777 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:27:22.984354 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:22.983899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56376)
0807 18:27:22.984151 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:22.984248 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260807 18:27:22.984676 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.004387 29859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 2
I20260807 18:27:23.005780 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.005121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56378)
0807 18:27:23.005391 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.005551 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:27:23.006937 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.006520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56390)
0807 18:27:23.006699 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.006791 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:23.007179 29859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.035920 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 1
I20260807 18:27:23.037309 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.036830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56400)
0807 18:27:23.037038 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.037138 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:27:23.038333 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.037974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56402)
0807 18:27:23.038142 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.038222 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:27:23.038559 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.098474 29859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 3
I20260807 18:27:23.100023 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.099296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56416)
0807 18:27:23.099623 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.099780 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:27:23.101476 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.100846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56428)
0807 18:27:23.101147 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.101266 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260807 18:27:23.101753 29859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.106783 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 2
I20260807 18:27:23.107863 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.107361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56434)
0807 18:27:23.107618 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.107702 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:27:23.108902 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.108555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56436)
0807 18:27:23.108715 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.108787 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:27:23.109197 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.215457 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 3
I20260807 18:27:23.216755 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.216260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56440)
0807 18:27:23.216468 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.216565 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 18:27:23.217871 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.217479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56456)
0807 18:27:23.217685 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.217760 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 18:27:23.218081 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
I20260807 18:27:23.245018 29859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 4
I20260807 18:27:23.246384 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.245757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56464)
0807 18:27:23.246028 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.246148 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:27:23.247638 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.247188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56468)
0807 18:27:23.247362 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.247469 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:27:23.247900 29859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:35557: Network error: Client connection negotiation failed: client connection to 127.29.156.252:35557: connect: Connection refused (error 111)
W20260807 18:27:23.286303 29871 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:23.286679 29871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:23.286809 29871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:23.286926 29871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:23.294479 29871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:23.294582 29871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:23.294674 29871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:23.294750 29871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:23.294838 29871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:23.306070 29871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:39187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:36767,127.29.156.253:43453,127.29.156.252:35557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:35557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.156.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:23.307503 29871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:23.308955 29871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:23.321758 29879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:23.322674 29881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:23.321786 29878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:23.323091 29871 server_base.cc:1034] running on GCE node
I20260807 18:27:23.324368 29871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:23.325923 29871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:23.327224 29871 hybrid_clock.cc:648] HybridClock initialized: now 1786127243327131 us; error 56 us; skew 500 ppm
I20260807 18:27:23.327680 29871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127243, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:23.331218 29871 init.cc:377] Logged in from keytab as kudu/127.29.156.252@KRBTEST.COM (short username kudu)
I20260807 18:27:23.332985 29871 webserver.cc:460] Webserver started at http://127.29.156.252:42659/ using document root <none> and password file <none>
I20260807 18:27:23.333720 29871 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:23.333849 29871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:23.334188 29871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:23.337342 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data/instance:
uuid: "53da09add5ef452ba591d4eb19f7b81b"
format_stamp: "Formatted at 2026-08-07 18:27:23 on dist-test-slave-sb2z"
server_key: "3719047643d3097d913b8a3f207cda24"
server_key_iv: "2c8ca879b22cc51384a65ce82badd2f6"
server_key_version: "encryptionkey@0"
I20260807 18:27:23.338191 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal/instance:
uuid: "53da09add5ef452ba591d4eb19f7b81b"
format_stamp: "Formatted at 2026-08-07 18:27:23 on dist-test-slave-sb2z"
server_key: "3719047643d3097d913b8a3f207cda24"
server_key_iv: "2c8ca879b22cc51384a65ce82badd2f6"
server_key_version: "encryptionkey@0"
I20260807 18:27:23.343636 29871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:27:23.347218 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:23.348661 29871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:27:23.348846 29871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "53da09add5ef452ba591d4eb19f7b81b"
format_stamp: "Formatted at 2026-08-07 18:27:23 on dist-test-slave-sb2z"
server_key: "3719047643d3097d913b8a3f207cda24"
server_key_iv: "2c8ca879b22cc51384a65ce82badd2f6"
server_key_version: "encryptionkey@0"
I20260807 18:27:23.349072 29871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:23.369499 29871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:23.374258 29871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:23.374624 29871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:23.394625 29871 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.252:35557
I20260807 18:27:23.394645 29975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.252:35557 every 8 connection(s)
I20260807 18:27:23.395308 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 4
I20260807 18:27:23.396310 29871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
I20260807 18:27:23.400803 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 29871
I20260807 18:27:23.401149 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126848590636-30323-0/minicluster-data/master-2/wal/instance
I20260807 18:27:23.401620 29976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:23.401741 30323 external_mini_cluster.cc:1428] Setting key 1d332e5c69f92357bb11a0150a56f00e
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:27:23.412154 29976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:27:23.421522 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.396115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56476)
0807 18:27:23.396371 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.396397 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:23.396545 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:23.402393 (+ 5848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:23.402399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:23.402935 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:23.403135 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.403147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.404898 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.404907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.407200 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.407215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.408346 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.408352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.408530 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.409173 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:23.409193 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:23.411702 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:23.416760 (+ 5058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.416769 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.416776 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.417566 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.418342 (+ 776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.418351 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.418357 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.418572 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.419998 (+ 1426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:23.420007 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:23.420270 (+ 263us) client_negotiation.cc:769] Sending connection context
0807 18:27:23.420463 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:27:23.420926 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 18:27:23.425457 29978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.397085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.29.156.252:35557)
0807 18:27:23.399331 (+ 2246us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.399344 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.399416 (+ 72us) server_negotiation.cc:407] Connection header received
0807 18:27:23.401503 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.401528 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.402017 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.402426 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.403321 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.404734 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.407343 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.408166 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.408386 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.412043 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.412054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.412076 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.412144 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.416547 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.417799 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.417812 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.417824 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.418157 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.418762 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.418769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.418775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.419756 (+ 981us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.422463 (+ 2707us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.422845 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.423518 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2048,"thread_start_us":1841,"threads_started":1}
I20260807 18:27:23.428393 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.403110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.29.156.252:35557)
0807 18:27:23.406500 (+ 3390us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.406506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.406529 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:27:23.406595 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.406600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.406669 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.406775 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.408763 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.410164 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.412050 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.412559 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.412710 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.417966 (+ 5256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.417976 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.417984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.418038 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.423023 (+ 4985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.424273 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.424282 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.424288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.424525 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.425109 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.425117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.425123 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.425602 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.425966 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.427416 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.427760 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3255,"thread_start_us":3073,"threads_started":1}
I20260807 18:27:23.435153 29745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: Bootstrap starting.
I20260807 18:27:23.439774 29745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: Neither blocks nor log segments found. Creating new log.
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127243, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:23.441334 29745 log.cc:824] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:23.445456 29745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: No bootstrap required, opened a new log
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:23.448758 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.415719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.29.156.254:36767)
0807 18:27:23.431077 (+ 15358us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.431083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.432095 (+ 1012us) server_negotiation.cc:407] Connection header received
0807 18:27:23.432154 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.432159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.432243 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.432356 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.433384 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.434558 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.436388 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.437226 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.437366 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.441363 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.441369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.441372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.441404 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.445430 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.446272 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.446277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.446279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.446443 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.446842 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.446847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.446851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.447184 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.447346 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.448019 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.448281 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15183,"thread_start_us":16250,"threads_started":1}
I20260807 18:27:23.448756 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.415856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:36767 (local address 127.0.0.1:60746)
0807 18:27:23.431691 (+ 15835us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.431729 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:23.431899 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:23.432421 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:23.432426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:23.432987 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:23.433224 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.433233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.434761 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.434765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.436173 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.436190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.437455 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.437463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.437685 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.438460 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:23.438483 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:23.441141 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:23.445618 (+ 4477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.445639 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.445648 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.446115 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.446552 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.446558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.446562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.446716 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.447330 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:23.447337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:23.447609 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 18:27:23.447844 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:27:23.448215 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15562,"thread_start_us":15371,"threads_started":1}
I20260807 18:27:23.450798 29976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:23.455240 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.436289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.29.156.254:36767)
0807 18:27:23.437216 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.437222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.437246 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:27:23.437302 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.437308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.437381 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.437566 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.439040 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.440015 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.441790 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.442666 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.442831 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.447650 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.447657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.447665 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.447707 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.451814 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.452781 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.452788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.452796 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.452978 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.453450 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.453456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.453459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.453866 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.454155 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.454512 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.454768 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":588,"threads_started":1}
I20260807 18:27:23.456948 29745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.457609 29745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f35f07f30eb4cdfae32835a9fee5d6d, State: Initialized, Role: FOLLOWER
I20260807 18:27:23.458361 29745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.459390 29990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.459933 29990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127243, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:23.460973 29745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:23.468648 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.451515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:43453 (local address 127.0.0.1:37974)
0807 18:27:23.451919 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.451939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:23.452050 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:23.452387 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:23.452392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:23.452886 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:23.453084 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.453097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.454496 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.454501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.456762 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.456781 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.457932 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.457936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.458075 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.458689 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:23.458713 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:23.461536 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:23.465532 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.465537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.465541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.466112 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.466593 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.466598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.466601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.466752 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.467568 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:23.467576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:23.467775 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 18:27:23.467941 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:27:23.468258 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262}
I20260807 18:27:23.469861 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.451726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.29.156.253:43453)
0807 18:27:23.451926 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.451934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.451989 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:27:23.452191 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.452196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.452264 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.452364 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.453274 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.454270 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.456952 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.457597 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.457671 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.461737 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.461742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.461745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.461805 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.465337 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.466249 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.466256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.466261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.466472 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.466877 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.466884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.466888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.467293 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.467470 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.469268 (+ 1798us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.469516 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:23.471069 29976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:23.479925 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.457078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.29.156.253:43453)
0807 18:27:23.458777 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.458785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.458811 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:27:23.458863 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.458868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.458940 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.460167 (+ 1227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.460856 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.462174 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.464535 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.465282 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.465445 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.471712 (+ 6267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.471722 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.471730 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.471771 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.475834 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.476739 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.476745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.476748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.476903 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.477362 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.477367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.477370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.477741 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.478012 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.479215 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.479445 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1568,"thread_start_us":1335,"threads_started":1}
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127243, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
W20260807 18:27:23.481788 30011 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:27:23.481982 30011 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:27:23.485445 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:23.487813 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.472062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56502)
0807 18:27:23.472789 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.472810 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:23.472921 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:23.473413 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:23.473419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:23.473963 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:23.474211 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.474225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.475674 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.475679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.477012 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.477027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.478119 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.478125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.478304 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.478916 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:23.478939 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:23.481556 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:23.485163 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.485168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.485171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.485585 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.485982 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.485988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.485991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.486122 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.486666 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:23.486670 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:23.486818 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 18:27:23.486965 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:27:23.487388 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":138}
W20260807 18:27:23.487769 29682 tablet.cc:2369] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:27:23.488683 29978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.472442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.29.156.252:35557)
0807 18:27:23.472646 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.472654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.473026 (+ 372us) server_negotiation.cc:407] Connection header received
0807 18:27:23.473087 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.473092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.473166 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.473256 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.474383 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.475490 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.477180 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.477811 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.477922 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.481750 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.481756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.481762 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.481798 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.485009 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.485699 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.485707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.485710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.485868 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.486225 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.486232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.486235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.486537 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.486716 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.487042 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.487263 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:27:23.489176 29713 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60748
I20260807 18:27:23.489588 29713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:27:23.489794 29713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:23.474251 29654 master_runner.cc:186] Error getting master registration for 127.29.156.253:43453: OK, instance_id { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" instance_seqno: 1786127242812979 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:27:23.492268 29654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:27:23.493080 29713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.493949 29976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: Bootstrap starting.
I20260807 18:27:23.495462 29713 leader_election.cc:288] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [CANDIDATE]: Term 1 election: Requested vote from peers d3fbd8689cc3495093f8bd6c361b4424 (127.29.156.253:43453), 53da09add5ef452ba591d4eb19f7b81b (127.29.156.252:35557)
I20260807 18:27:23.498356 29976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:23.499945 29976 log.cc:824] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:23.508337 29976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: No bootstrap required, opened a new log
I20260807 18:27:23.522997 29976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.523594 29976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53da09add5ef452ba591d4eb19f7b81b, State: Initialized, Role: FOLLOWER
I20260807 18:27:23.524277 29976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.531391 30024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.531921 30024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.534032 29976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:23.534155 29932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "53da09add5ef452ba591d4eb19f7b81b"
I20260807 18:27:23.534745 29932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:23.538780 29932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f35f07f30eb4cdfae32835a9fee5d6d in term 1.
I20260807 18:27:23.539908 29678 leader_election.cc:302] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53da09add5ef452ba591d4eb19f7b81b, 9f35f07f30eb4cdfae32835a9fee5d6d; no voters:
I20260807 18:27:23.540436 29990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:23.547122 29859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } attempt: 5
I20260807 18:27:23.557711 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.519217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.29.156.254:36767)
0807 18:27:23.519413 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.519419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.526649 (+ 7230us) server_negotiation.cc:407] Connection header received
0807 18:27:23.527177 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.527187 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.527310 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.527401 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.537393 (+ 9992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.538891 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.543872 (+ 4981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.544879 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.545022 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.548849 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.548859 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.548866 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.548935 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.553398 (+ 4463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.554424 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.554434 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.554441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.554631 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.555218 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.555226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.555232 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.555710 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.556039 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.556799 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.557163 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 18:27:23 dist-test-slave-sb2z krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127242, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:27:23.564930 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.513579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.29.156.253:43453)
0807 18:27:23.513783 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.513792 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.515053 (+ 1261us) server_negotiation.cc:407] Connection header received
0807 18:27:23.515663 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.515670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.515758 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.515858 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.521589 (+ 5731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.522724 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.528254 (+ 5530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.528912 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.529024 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.544928 (+ 15904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.544939 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.544946 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.545012 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.555425 (+ 10413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.561967 (+ 6542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.561976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.561980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.562165 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.562936 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.562941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.562944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.563374 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.564097 (+ 723us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.564246 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.564510 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:27:23.566540 29990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [term 1 LEADER]: Becoming Leader. State: Replica: 9f35f07f30eb4cdfae32835a9fee5d6d, State: Running, Role: LEADER
I20260807 18:27:23.567284 29990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.567658 29978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.514581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.29.156.252:35557)
0807 18:27:23.514785 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.514793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.525982 (+ 11189us) server_negotiation.cc:407] Connection header received
0807 18:27:23.530185 (+ 4203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.530195 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.530310 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.530402 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.533775 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.534907 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.541650 (+ 6743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.542496 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.542634 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.553452 (+ 10818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.553457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.553461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.553501 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.559479 (+ 5978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.563213 (+ 3734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.563218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.563222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.563382 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.565299 (+ 1917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.565304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.565307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.565832 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.566250 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.566968 (+ 718us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.567234 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":26880}
I20260807 18:27:23.571960 30026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f35f07f30eb4cdfae32835a9fee5d6d. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.574014 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.547813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:35557 (local address 127.0.0.1:56524)
0807 18:27:23.548228 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:23.548251 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:23.548369 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:23.549289 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:23.549295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:23.549829 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:23.550072 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.550089 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.551766 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.551772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.553265 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:23.553283 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.554527 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.554534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:23.554784 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.558353 (+ 3569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:23.558380 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:23.564510 (+ 6130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:23.570946 (+ 6436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.570954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.570960 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.571494 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.572059 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:23.572064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:23.572068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:23.572294 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:23.572983 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:23.572988 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:23.573178 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:27:23.573312 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 18:27:23.573636 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260807 18:27:23.574766 30026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:23.575151 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:23.548603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.29.156.252:35557)
0807 18:27:23.548817 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.548823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.548842 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:27:23.548897 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.548903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.548979 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.549090 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.550274 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.551476 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.553452 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:23.554170 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:23.554308 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:23.564970 (+ 10662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:23.564979 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:23.565274 (+ 295us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:23.565348 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:23.570778 (+ 5430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.571733 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.571738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.571741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.571910 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:23.572460 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:23.572466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:23.572471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:23.572831 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:23.574419 (+ 1588us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:23.574570 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 18:27:23.574813 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:27:23.581664 29859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: Bootstrap starting.
I20260807 18:27:23.584478 30036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:23.586275 29859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:23.587857 29859 log.cc:824] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:23.592550 29859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: No bootstrap required, opened a new log
I20260807 18:27:23.592253 30036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:23.605803 29859 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.606334 29859 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3fbd8689cc3495093f8bd6c361b4424, State: Initialized, Role: FOLLOWER
I20260807 18:27:23.606959 29859 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } }
I20260807 18:27:23.609196 30045 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:27:23.609858 30043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.610505 30043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.611639 29818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d3fbd8689cc3495093f8bd6c361b4424"
I20260807 18:27:23.612216 29818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:23.616780 29818 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f35f07f30eb4cdfae32835a9fee5d6d in term 1.
I20260807 18:27:23.611631 29859 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:27:23.627050 29903 tablet.cc:2369] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:27:23.616413 29871 master_runner.cc:186] Error getting master registration for 127.29.156.253:43453: OK, instance_id { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" instance_seqno: 1786127242812979 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:27:23.631184 29871 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:27:23.632550 29932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [term 1 FOLLOWER]: Refusing update from remote peer 9f35f07f30eb4cdfae32835a9fee5d6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:27:23.632929 29818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [term 1 FOLLOWER]: Refusing update from remote peer 9f35f07f30eb4cdfae32835a9fee5d6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:27:23.634029 30026 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:27:23.634406 29990 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:27:23.654789 30043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f35f07f30eb4cdfae32835a9fee5d6d. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.655181 30043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.659106 30046 mvcc.cc:205] Tried to move back new op lower bound from 7315977189908484096 to 7315977189658030080. Current Snapshot: MvccSnapshot[applied={T|T < 7315977189908484096}]
I20260807 18:27:23.661372 29990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.661778 29990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:23.662426 30043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.662704 30043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.665542 30026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.665840 30026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:23.668676 30036 catalog_manager.cc:1347] Generated new cluster ID: b76afe8807a742c6baac16ca333a0f71
I20260807 18:27:23.668781 30036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:23.696580 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.697166 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.697633 30024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f35f07f30eb4cdfae32835a9fee5d6d. Latest consensus state: current_term: 1 leader_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f35f07f30eb4cdfae32835a9fee5d6d" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 36767 } } peers { permanent_uuid: "d3fbd8689cc3495093f8bd6c361b4424" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 43453 } } peers { permanent_uuid: "53da09add5ef452ba591d4eb19f7b81b" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 35557 } } }
I20260807 18:27:23.697938 30024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:23.716274 30075 catalog_manager.cc:1259] Loaded cluster ID: b76afe8807a742c6baac16ca333a0f71
I20260807 18:27:23.716475 30075 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: loading cluster ID for follower catalog manager: success
W20260807 18:27:23.717880 30075 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260807 18:27:23.718115 30075 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:27:23.745146 30036 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:23.747491 30036 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:23.762738 30036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f35f07f30eb4cdfae32835a9fee5d6d: Generated new TSK 0
I20260807 18:27:23.766326 30036 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:27:23.896407 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.893248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38006 (local address 127.29.156.253:43453)
0807 18:27:23.893436 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.893482 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.894473 (+ 991us) server_negotiation.cc:407] Connection header received
0807 18:27:23.894643 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.894651 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.894753 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.894886 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.896104 (+ 1218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38006: BlockingRecv error: recv got EOF from 127.0.0.1:38006 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:27:23.897780 29981 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.894748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.29.156.252:35557)
0807 18:27:23.894952 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.894959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.896184 (+ 1225us) server_negotiation.cc:407] Connection header received
0807 18:27:23.896343 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.896348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.896441 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.896549 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.897533 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56534: BlockingRecv error: recv got EOF from 127.0.0.1:56534 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 18:27:23.902732 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.900742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.29.156.254:36767)
0807 18:27:23.900946 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.900953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.901405 (+ 452us) server_negotiation.cc:407] Connection header received
0807 18:27:23.901587 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.901593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.901681 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.901777 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.902486 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60760: BlockingRecv error: recv got EOF from 127.0.0.1:60760 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 18:27:23.910347 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.907871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.29.156.253:43453)
0807 18:27:23.908062 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.908068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.908519 (+ 451us) server_negotiation.cc:407] Connection header received
0807 18:27:23.908834 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.908842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.909003 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.909084 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.910118 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38012: BlockingRecv error: recv got EOF from 127.0.0.1:38012 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:27:23.918493 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.916012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.29.156.253:43453)
0807 18:27:23.916208 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.916214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.916676 (+ 462us) server_negotiation.cc:407] Connection header received
0807 18:27:23.916973 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.916980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.917067 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.917140 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.918181 (+ 1041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38028: BlockingRecv error: recv got EOF from 127.0.0.1:38028 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 18:27:23.927572 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.925014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.29.156.253:43453)
0807 18:27:23.925192 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.925201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.925742 (+ 541us) server_negotiation.cc:407] Connection header received
0807 18:27:23.926057 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.926064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.926154 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.926236 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.927282 (+ 1046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38030: BlockingRecv error: recv got EOF from 127.0.0.1:38030 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 18:27:23.938638 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.936104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.29.156.253:43453)
0807 18:27:23.936328 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.936334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.936822 (+ 488us) server_negotiation.cc:407] Connection header received
0807 18:27:23.937160 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.937166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.937254 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.937338 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.938377 (+ 1039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38038: BlockingRecv error: recv got EOF from 127.0.0.1:38038 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260807 18:27:23.956184 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.953683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.29.156.253:43453)
0807 18:27:23.953852 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.953859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.954431 (+ 572us) server_negotiation.cc:407] Connection header received
0807 18:27:23.954667 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.954675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.954798 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.954924 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.955887 (+ 963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38042: BlockingRecv error: recv got EOF from 127.0.0.1:38042 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 18:27:23.979229 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:23.976599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.29.156.253:43453)
0807 18:27:23.976822 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:23.976829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:23.977456 (+ 627us) server_negotiation.cc:407] Connection header received
0807 18:27:23.977703 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:23.977710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:23.977819 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:23.977948 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:23.978915 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38044: BlockingRecv error: recv got EOF from 127.0.0.1:38044 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 18:27:24.024833 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:24.022099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.29.156.253:43453)
0807 18:27:24.022308 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:24.022316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:24.022973 (+ 657us) server_negotiation.cc:407] Connection header received
0807 18:27:24.023214 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:24.023221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:24.023341 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:24.023483 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:24.024528 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38052: BlockingRecv error: recv got EOF from 127.0.0.1:38052 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 18:27:24.095238 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:24.092580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.29.156.253:43453)
0807 18:27:24.092851 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:24.092857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:24.093331 (+ 474us) server_negotiation.cc:407] Connection header received
0807 18:27:24.093650 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:24.093655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:24.093742 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:24.093853 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:24.094908 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38068: BlockingRecv error: recv got EOF from 127.0.0.1:38068 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 18:27:24.229135 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:24.227143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.29.156.253:43453)
0807 18:27:24.227404 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:24.227411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:24.227798 (+ 387us) server_negotiation.cc:407] Connection header received
0807 18:27:24.227934 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:24.227938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:24.228013 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:24.228107 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:24.228862 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38070: BlockingRecv error: recv got EOF from 127.0.0.1:38070 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 18:27:24.494042 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:24.492066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.29.156.253:43453)
0807 18:27:24.492292 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:24.492299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:24.492424 (+ 125us) server_negotiation.cc:407] Connection header received
0807 18:27:24.492713 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:24.492718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:24.492807 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:24.492920 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:24.493744 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38078: BlockingRecv error: recv got EOF from 127.0.0.1:38078 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:27:24.615118 30045 catalog_manager.cc:1259] Loaded cluster ID: b76afe8807a742c6baac16ca333a0f71
I20260807 18:27:24.615289 30045 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: loading cluster ID for follower catalog manager: success
I20260807 18:27:24.618474 30045 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: acquiring CA information for follower catalog manager: success
I20260807 18:27:24.620638 30045 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 53da09add5ef452ba591d4eb19f7b81b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:27:24.723345 30075 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: acquiring CA information for follower catalog manager: success
I20260807 18:27:24.726173 30075 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3fbd8689cc3495093f8bd6c361b4424: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:27:25.017184 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:25.010095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.29.156.253:43453)
0807 18:27:25.011027 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:25.011032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:25.011052 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:27:25.011171 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:25.011175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:25.011360 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:25.011490 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:25.012142 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.012886 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.014516 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.014985 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.015102 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:25.015828 (+ 726us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:25.016478 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:25.016626 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:25.016831 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 18:27:25.016922 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":706,"threads_started":1}
I20260807 18:27:25.031607 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:25.024676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56542 (local address 127.29.156.252:35557)
0807 18:27:25.025280 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:25.025285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:25.025333 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:27:25.025618 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:25.025622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:25.025795 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:25.025958 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:25.026446 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.027239 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.028791 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.029249 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.029355 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:25.030050 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:25.030735 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:25.030863 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:25.031008 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 18:27:25.031089 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":302,"threads_started":1}
I20260807 18:27:25.044941 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:25.039051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.29.156.252:35557)
0807 18:27:25.039789 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:25.039798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:25.040008 (+ 210us) server_negotiation.cc:407] Connection header received
0807 18:27:25.040226 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:25.040232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:25.040313 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:25.040416 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:25.040932 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.041696 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.043125 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.043612 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.043747 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:25.044166 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:25.044300 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:25.044398 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:25.044548 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:27:25.044630 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560}
I20260807 18:27:25.050830 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:25.039126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.29.156.253:43453)
0807 18:27:25.039386 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:25.039396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:25.045003 (+ 5607us) server_negotiation.cc:407] Connection header received
0807 18:27:25.045445 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:25.045452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:25.045560 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:25.045630 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:25.046251 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.047130 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.048875 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.049534 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.049661 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:25.050126 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:25.050238 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:25.050309 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:25.050467 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 18:27:25.050535 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:27:25.054323 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:25.039395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.29.156.254:36767)
0807 18:27:25.047978 (+ 8583us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:25.047985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:25.048438 (+ 453us) server_negotiation.cc:407] Connection header received
0807 18:27:25.048638 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:25.048646 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:25.048878 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:25.049044 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:25.049621 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.050689 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.052221 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:25.052701 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:25.052835 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:25.053325 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:25.053730 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:25.053853 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:25.053953 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 18:27:25.054047 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8426,"thread_start_us":8312,"threads_started":1}
I20260807 18:27:25.060824 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29654
I20260807 18:27:25.062155 29736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:25.178174 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29654
I20260807 18:27:25.210295 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29760
I20260807 18:27:25.211642 29852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:25.323964 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29760
I20260807 18:27:25.352381 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 29871
I20260807 18:27:25.353806 29969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:25.463284 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 29871
2026-08-07T18:27:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5441 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:27:25 dist-test-slave-sb2z krb5kdc[30194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:27:25 dist-test-slave-sb2z krb5kdc[30194](info): set up 2 sockets
Aug 07 18:27:25 dist-test-slave-sb2z krb5kdc[30194](info): commencing operation
krb5kdc: starting...
W20260807 18:27:26.841056 30323 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/test-user.keytab.
Aug 07 18:27:27 dist-test-slave-sb2z krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:27:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:27:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.156.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
Entry for principal kudu/127.29.156.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab.
I20260807 18:27:27.220860 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44409
--webserver_interface=127.29.156.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:27:27.582047 30210 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:27.582404 30210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:27.582542 30210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:27.582682 30210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:27.589979 30210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:27.590083 30210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:27.590184 30210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:27.590266 30210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:27.590359 30210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:27.600328 30210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.254.keytab
--principal=kudu/127.29.156.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.254:44409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.156.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:27.601697 30210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:27.603139 30210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:27.615789 30218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:27.615870 30217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:27.615844 30221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:27.616746 30210 server_base.cc:1034] running on GCE node
I20260807 18:27:27.617604 30210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:27.619056 30210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:27.620338 30210 hybrid_clock.cc:648] HybridClock initialized: now 1786127247620255 us; error 46 us; skew 500 ppm
I20260807 18:27:27.620719 30210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:27 dist-test-slave-sb2z krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:27.624327 30210 init.cc:377] Logged in from keytab as kudu/127.29.156.254@KRBTEST.COM (short username kudu)
I20260807 18:27:27.625967 30210 webserver.cc:460] Webserver started at http://127.29.156.254:33241/ using document root <none> and password file <none>
I20260807 18:27:27.626711 30210 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:27.626853 30210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:27.627204 30210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:27.630528 30210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data/instance:
uuid: "b92d718307eb426bbfd8aeaa1f964498"
format_stamp: "Formatted at 2026-08-07 18:27:27 on dist-test-slave-sb2z"
server_key: "c10ad3b905fc5e229b67f32a81ef8104"
server_key_iv: "56fa2a84f9ccf5fcd0336bfc754c6ac1"
server_key_version: "encryptionkey@0"
I20260807 18:27:27.631385 30210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance:
uuid: "b92d718307eb426bbfd8aeaa1f964498"
format_stamp: "Formatted at 2026-08-07 18:27:27 on dist-test-slave-sb2z"
server_key: "c10ad3b905fc5e229b67f32a81ef8104"
server_key_iv: "56fa2a84f9ccf5fcd0336bfc754c6ac1"
server_key_version: "encryptionkey@0"
I20260807 18:27:27.636834 30210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:27:27.640648 30231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:27.642158 30210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:27:27.642380 30210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal
uuid: "b92d718307eb426bbfd8aeaa1f964498"
format_stamp: "Formatted at 2026-08-07 18:27:27 on dist-test-slave-sb2z"
server_key: "c10ad3b905fc5e229b67f32a81ef8104"
server_key_iv: "56fa2a84f9ccf5fcd0336bfc754c6ac1"
server_key_version: "encryptionkey@0"
I20260807 18:27:27.642632 30210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:27.664830 30210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:27.669678 30210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:27.670079 30210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:27.690344 30210 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.254:44409
I20260807 18:27:27.690388 30306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.254:44409 every 8 connection(s)
I20260807 18:27:27.691859 30210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/data/info.pb
I20260807 18:27:27.695982 30308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:27.698648 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30210
I20260807 18:27:27.698843 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-0/wal/instance
I20260807 18:27:27.699342 30323 external_mini_cluster.cc:1428] Setting key eb20f9932fd67408b14dd900abc5ab2e
I20260807 18:27:27.703413 30308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:27 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:27.731194 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.706967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44409 (local address 127.0.0.1:60626)
0807 18:27:27.707936 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.707978 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:27.710270 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:27.711174 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:27.711181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:27.711981 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:27.712345 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:27.712363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:27.714120 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:27.714125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:27.715753 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:27.715774 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:27.716824 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:27.716836 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:27.717154 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:27.717968 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:27.718006 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:27.721100 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:27.726463 (+ 5363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:27.726475 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:27.726486 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:27.727034 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:27.727636 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:27.727640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:27.727642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:27.727812 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:27.729087 (+ 1275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:27.729094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:27.729316 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 18:27:27.729555 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:27:27.730043 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":357,"threads_started":1}
I20260807 18:27:27.731266 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.708320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.29.156.254:44409)
0807 18:27:27.709149 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:27.709158 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:27.709218 (+ 60us) server_negotiation.cc:407] Connection header received
0807 18:27:27.710670 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:27.710676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:27.710930 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:27.711160 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:27.712580 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:27.713905 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:27.715986 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:27.716552 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:27.716697 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:27.721416 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:27.721430 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:27.721452 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:27.721502 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:27.726266 (+ 4764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:27.727223 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:27.727236 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:27.727243 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:27.727477 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:27.727951 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:27.727959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:27.727966 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:27.728908 (+ 942us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:27.729161 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:27.729787 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 18:27:27.730329 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":526,"thread_start_us":353,"threads_started":1}
I20260807 18:27:27.734493 30308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:27.735937 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.735302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43028)
0807 18:27:27.735565 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.735738 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 18:27:27.736232 30235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
I20260807 18:27:27.738225 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.737581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43034)
0807 18:27:27.737791 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.737903 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:27:27.738468 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:42959: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.156.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
Entry for principal kudu/127.29.156.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab.
I20260807 18:27:27.787065 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:42959
--webserver_interface=127.29.156.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:27:27.787875 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } attempt: 1
I20260807 18:27:27.789357 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.788761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43038)
0807 18:27:27.789028 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.789199 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 18:27:27.790580 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.790160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43040)
0807 18:27:27.790353 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.790431 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 18:27:27.790793 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:42959: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
I20260807 18:27:27.859140 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } attempt: 2
I20260807 18:27:27.860678 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.860004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43046)
0807 18:27:27.860322 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.860458 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 18:27:27.861946 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.861450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43052)
0807 18:27:27.861643 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.861760 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 18:27:27.862216 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:42959: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
I20260807 18:27:27.953527 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } attempt: 3
I20260807 18:27:27.954865 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.954317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43066)
0807 18:27:27.954577 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.954696 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 18:27:27.956200 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:27.955707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43068)
0807 18:27:27.955879 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:27.955992 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:27:27.956455 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:42959: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
I20260807 18:27:28.099759 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } attempt: 4
I20260807 18:27:28.101235 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.100513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43080)
0807 18:27:28.100908 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.101036 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260807 18:27:28.102521 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.102069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43088)
0807 18:27:28.102242 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.102349 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:28.102784 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.253:42959: Network error: Client connection negotiation failed: client connection to 127.29.156.253:42959: connect: Connection refused (error 111)
W20260807 18:27:28.160681 30338 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:28.161041 30338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:28.161123 30338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:28.161208 30338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:28.168526 30338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:28.168623 30338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:28.168684 30338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:28.168737 30338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:28.168803 30338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:28.178929 30338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.253.keytab
--principal=kudu/127.29.156.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.253:42959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.156.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:28.180248 30338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:28.181545 30338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:28.194320 30352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:28.194469 30357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:28.194428 30354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:28.195681 30338 server_base.cc:1034] running on GCE node
I20260807 18:27:28.196681 30338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:28.198242 30338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:28.199599 30338 hybrid_clock.cc:648] HybridClock initialized: now 1786127248199476 us; error 64 us; skew 500 ppm
I20260807 18:27:28.199985 30338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:28.203405 30338 init.cc:377] Logged in from keytab as kudu/127.29.156.253@KRBTEST.COM (short username kudu)
I20260807 18:27:28.205015 30338 webserver.cc:460] Webserver started at http://127.29.156.253:40261/ using document root <none> and password file <none>
I20260807 18:27:28.205716 30338 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:28.205806 30338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:28.206085 30338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:28.209299 30338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data/instance:
uuid: "63ea1e59080344928799582cbd50972b"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "ee7b546a513d0795d349a54aa281fc17"
server_key_iv: "271c706941e68f64e4ee447952c28b6c"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.210124 30338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal/instance:
uuid: "63ea1e59080344928799582cbd50972b"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "ee7b546a513d0795d349a54aa281fc17"
server_key_iv: "271c706941e68f64e4ee447952c28b6c"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.215696 30338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 18:27:28.219230 30367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:28.220677 30338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:27:28.220829 30338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal
uuid: "63ea1e59080344928799582cbd50972b"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "ee7b546a513d0795d349a54aa281fc17"
server_key_iv: "271c706941e68f64e4ee447952c28b6c"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.221113 30338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:28.249610 30338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:28.254483 30338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:28.254792 30338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:28.275123 30338 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.253:42959
I20260807 18:27:28.275250 30438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.253:42959 every 8 connection(s)
I20260807 18:27:28.276608 30338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/data/info.pb
I20260807 18:27:28.277881 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30338
I20260807 18:27:28.278090 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-1/wal/instance
I20260807 18:27:28.278573 30323 external_mini_cluster.cc:1428] Setting key c4517e407b172dbff9638f6088abd63d
I20260807 18:27:28.280726 30439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:28.287422 30439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:28.313495 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.290244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.29.156.254:44409)
0807 18:27:28.290877 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.290884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.291874 (+ 990us) server_negotiation.cc:407] Connection header received
0807 18:27:28.294995 (+ 3121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.295004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.295104 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.295243 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.297005 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.298348 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.300138 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.300883 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.301051 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.305076 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.305086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.305094 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.305154 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.309277 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.310125 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.310133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.310140 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.310306 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.310781 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.310789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.310795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.311247 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.311414 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.312685 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.312968 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":334,"threads_started":1}
I20260807 18:27:28.314409 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.290758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44409 (local address 127.0.0.1:60632)
0807 18:27:28.291667 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.291723 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.294662 (+ 2939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.295469 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.295479 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.296187 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.296766 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.296784 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.298550 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.298554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.299933 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.299988 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.301174 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.301179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.301394 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.302249 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.302275 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.304763 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.309455 (+ 4692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.309463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.309483 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.309971 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.310422 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.310426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.310428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.310642 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.311673 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.311680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.312090 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.312521 (+ 431us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.313045 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":323,"threads_started":1}
I20260807 18:27:28.316608 30439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.156.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.156.252@KRBTEST.COM" created.
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:28.332535 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.317333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43092)
0807 18:27:28.317526 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.317548 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.317654 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.318806 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.318810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.319251 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.319396 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.319404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.321467 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.321475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.322445 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.322457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.323319 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.323324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.323605 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.324274 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.324301 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.326480 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.329815 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.329820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.329824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.330244 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.330634 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.330639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.330642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.330792 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.331394 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.331403 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.331573 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.331728 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.332035 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":22}
I20260807 18:27:28.332583 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.317647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.29.156.253:42959)
0807 18:27:28.318381 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.318387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.318414 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:27:28.318457 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.318462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.318648 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.318763 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.319645 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.321344 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.322595 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.323169 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.323310 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.326692 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.326703 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.326716 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.326750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.329663 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.330383 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.330395 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.330397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.330509 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.330895 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.330903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.330906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.331264 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.331496 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.331885 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.332145 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":586,"thread_start_us":430,"threads_started":1}
I20260807 18:27:28.334909 30439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:28.336334 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.335762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32868)
0807 18:27:28.335969 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.336099 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:27:28.336659 30373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.338523 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.338008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32880)
0807 18:27:28.338258 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.338366 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 18:27:28.338763 30439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
Entry for principal kudu/127.29.156.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab.
I20260807 18:27:28.364117 30323 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv84zy/build/asan/bin/kudu
/tmp/dist-test-taskyv84zy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:33471
--webserver_interface=127.29.156.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.156.212:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:27:28.388156 30439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 1
I20260807 18:27:28.389370 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.388897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32884)
0807 18:27:28.389116 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.389219 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:27:28.390513 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.390150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32890)
0807 18:27:28.390331 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.390411 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:28.390756 30439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.402060 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } attempt: 5
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:28.416481 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.402777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43094)
0807 18:27:28.402999 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.403020 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.403148 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.403517 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.403541 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.404079 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.404303 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.404314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.405289 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.405297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.406742 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.406758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.407500 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.407507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.407778 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.408381 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.408404 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.410860 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.413974 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.413980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.413984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.414368 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.414750 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.414755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.414759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.414900 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.415428 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.415433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.415643 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.415801 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.416126 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 18:27:28.416473 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.402861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.29.156.253:42959)
0807 18:27:28.403043 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.403048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.403075 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:27:28.403295 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.403299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.403365 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.403452 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.404457 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.405164 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.406905 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.407353 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.407449 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.411059 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.411064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.411068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.411100 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.413823 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.414487 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.414494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.414497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.414633 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.415012 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.415017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.415021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.415304 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.415462 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.415910 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.416140 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:27:28.417709 30308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:28.418839 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.418452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32892)
0807 18:27:28.418626 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.418718 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:27:28.419961 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.419629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32894)
0807 18:27:28.419781 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.419858 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:27:28.420205 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.459116 30439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 2
I20260807 18:27:28.460632 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.460088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32906)
0807 18:27:28.460327 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.460428 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 18:27:28.461797 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.461400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32922)
0807 18:27:28.461586 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.461661 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:27:28.462033 30439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.471455 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 1
I20260807 18:27:28.472858 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.472197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32930)
0807 18:27:28.472436 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.472552 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:27:28.473942 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.473552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32936)
0807 18:27:28.473717 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.473805 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:28.474263 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.542539 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 2
I20260807 18:27:28.544081 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.543424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32944)
0807 18:27:28.543735 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.543865 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 18:27:28.545400 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.544924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32960)
0807 18:27:28.545113 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.545225 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 18:27:28.545729 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.553256 30439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 3
I20260807 18:27:28.554433 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.553899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32972)
0807 18:27:28.554124 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.554236 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 18:27:28.555735 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.555181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32988)
0807 18:27:28.555476 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.555586 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260807 18:27:28.556015 30439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.651978 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 3
I20260807 18:27:28.653374 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.652739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:32992)
0807 18:27:28.653010 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.653135 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:27:28.654726 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.654222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33002)
0807 18:27:28.654467 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.654569 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260807 18:27:28.655006 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.699275 30439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 4
I20260807 18:27:28.700516 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.699987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33006)
0807 18:27:28.700250 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.700353 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 18:27:28.701581 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.701241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33022)
0807 18:27:28.701397 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.701481 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:27:28.701892 30439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
W20260807 18:27:28.734977 30456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:27:28.735363 30456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:27:28.735497 30456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:27:28.735661 30456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:27:28.743108 30456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:27:28.743203 30456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:27:28.743299 30456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:27:28.743377 30456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:27:28.743474 30456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:27:28.753871 30456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.156.212:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.156.254:44409,127.29.156.253:42959,127.29.156.252:33471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/krb5kdc/kudu_127.29.156.252.keytab
--principal=kudu/127.29.156.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.156.252:33471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.156.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:27:28.755290 30456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:27:28.756721 30456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:27:28.769467 30471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:28.769712 30466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:27:28.769652 30468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:27:28.771189 30456 server_base.cc:1034] running on GCE node
I20260807 18:27:28.772190 30456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:27:28.773923 30456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:27:28.775213 30456 hybrid_clock.cc:648] HybridClock initialized: now 1786127248775130 us; error 46 us; skew 500 ppm
I20260807 18:27:28.775755 30456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:27:28.779649 30456 init.cc:377] Logged in from keytab as kudu/127.29.156.252@KRBTEST.COM (short username kudu)
I20260807 18:27:28.781396 30456 webserver.cc:460] Webserver started at http://127.29.156.252:35577/ using document root <none> and password file <none>
I20260807 18:27:28.782163 30456 fs_manager.cc:359] Metadata directory not provided
I20260807 18:27:28.782303 30456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:27:28.782661 30456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:27:28.785866 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data/instance:
uuid: "7df4874115cc4447a8dd0d5ba029dc44"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "00a5dca1e4ac8e2de760de43d4972cd0"
server_key_iv: "7308c0ba9982949da168925b64c0162f"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.786723 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal/instance:
uuid: "7df4874115cc4447a8dd0d5ba029dc44"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "00a5dca1e4ac8e2de760de43d4972cd0"
server_key_iv: "7308c0ba9982949da168925b64c0162f"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.792598 30456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 18:27:28.796612 30482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:27:28.798316 30456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:27:28.798542 30456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal
uuid: "7df4874115cc4447a8dd0d5ba029dc44"
format_stamp: "Formatted at 2026-08-07 18:27:28 on dist-test-slave-sb2z"
server_key: "00a5dca1e4ac8e2de760de43d4972cd0"
server_key_iv: "7308c0ba9982949da168925b64c0162f"
server_key_version: "encryptionkey@0"
I20260807 18:27:28.798794 30456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:27:28.817548 30456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:27:28.822130 30456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:27:28.822427 30456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:27:28.832280 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 4
I20260807 18:27:28.833700 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.833074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33036)
0807 18:27:28.833364 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.833489 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:27:28.834935 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.834434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33038)
0807 18:27:28.834638 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.834744 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:27:28.835265 30308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.156.252:33471: Network error: Client connection negotiation failed: client connection to 127.29.156.252:33471: connect: Connection refused (error 111)
I20260807 18:27:28.843008 30456 rpc_server.cc:307] RPC server started. Bound to: 127.29.156.252:33471
I20260807 18:27:28.843030 30549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.156.252:33471 every 8 connection(s)
I20260807 18:27:28.844521 30456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/data/info.pb
I20260807 18:27:28.848683 30550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:27:28.852511 30323 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv84zy/build/asan/bin/kudu as pid 30456
I20260807 18:27:28.852733 30323 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv84zy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126848590636-30323-0/minicluster-data/master-2/wal/instance
I20260807 18:27:28.853190 30323 external_mini_cluster.cc:1428] Setting key 2a8ff68bce86a407cd4af469febd06fa
I20260807 18:27:28.856257 30550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:28.876761 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.860390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.29.156.254:44409)
0807 18:27:28.861546 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.861551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.861567 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:27:28.861614 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.861617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.861683 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.861812 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.863054 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.863835 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.865272 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.865682 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.865821 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.869853 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.869858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.869861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.869903 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.873159 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.874161 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.874166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.874169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.874321 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.874732 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.874737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.874740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.875170 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.875326 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.876086 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.876332 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":956,"threads_started":1}
I20260807 18:27:28.877542 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.860285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.254:44409 (local address 127.0.0.1:60636)
0807 18:27:28.860972 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.861022 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.861166 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.861890 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.861914 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.862566 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.862875 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.862884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.864007 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.864011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.865107 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.865119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.865932 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.865941 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.866101 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.866753 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.866781 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.869641 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.873311 (+ 3670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.873326 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.873334 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.873979 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.874479 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.874482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.874485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.874618 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.875334 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.875342 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.875649 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.875910 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.876269 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":2,"thread_start_us":290,"threads_started":1}
I20260807 18:27:28.877557 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.855208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.29.156.252:33471)
0807 18:27:28.856042 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.856054 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.856098 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:27:28.858270 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.858295 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.858749 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.859075 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.861184 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.862645 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.864018 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.864730 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.864961 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.869487 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.869496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.869515 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.869553 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.872565 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.873297 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.873303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.873310 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.873476 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.874160 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.874166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.874169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.874916 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.875253 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.875739 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.876421 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":608,"thread_start_us":401,"threads_started":1}
I20260807 18:27:28.879657 30550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.254@KRBTEST.COM
I20260807 18:27:28.895311 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.880334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.253:42959 (local address 127.0.0.1:43098)
0807 18:27:28.880516 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.880539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.880644 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.881299 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.881303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.881794 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.881974 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.881988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.883505 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.883509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.884891 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.884905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.885912 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.885919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.886088 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.886835 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.886863 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.889316 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.892827 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.892832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.892836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.893225 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.893630 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.893635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.893638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.893786 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.894304 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.894308 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.894467 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.894626 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.894927 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:27:28.895341 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.880696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.29.156.253:42959)
0807 18:27:28.880943 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.880951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.880975 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:27:28.881039 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.881044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.881131 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.881217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.882164 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.883326 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.885048 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.885727 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.885866 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.889524 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.889530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.889537 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.889588 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.892676 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.893348 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.893355 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.893358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.893512 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.893899 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.893904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.893910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.894179 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.894342 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.894719 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.894944 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:27:28.896430 30550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:27:28.899116 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.881836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.29.156.254:44409)
0807 18:27:28.882115 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.882123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.882142 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:27:28.882190 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.882193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.882256 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.882345 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.884009 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.885027 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.886800 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.887540 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.887682 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.892562 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.892568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.892571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.892606 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.895788 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.896672 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.896680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.896684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.896841 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.897437 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.897442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.897445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.897817 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.898113 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.898463 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.898715 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.252@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
Aug 07 18:27:28 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.156.253@KRBTEST.COM
I20260807 18:27:28.911412 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.897434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.29.156.252:33471)
0807 18:27:28.897645 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.897652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.897670 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:27:28.897838 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.897842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.897933 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.898027 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.899001 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.900042 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.901922 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.902541 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.902675 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.906124 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.906129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.906132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.906160 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.908854 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.909523 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.909530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.909533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.909659 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.910011 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.910017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.910020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.910276 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.910435 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.910778 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.911029 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260807 18:27:28.911633 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.897419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33060)
0807 18:27:28.897606 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:28.897625 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:28.897728 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:28.898095 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:28.898100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:28.898661 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:28.898847 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.898860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.900209 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.900221 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.901732 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:28.901746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.902852 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.902859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:28.903086 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.903899 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:28.903925 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:28.905930 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:28.909024 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.909030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.909034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.909400 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.909763 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:28.909769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:28.909773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:28.909901 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:28.910387 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:28.910392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:28.910560 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 18:27:28.910702 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:27:28.911206 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":140}
I20260807 18:27:28.917500 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:28.902201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.29.156.253:42959)
0807 18:27:28.902501 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:28.902509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:28.902530 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:27:28.902599 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:28.902607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:28.902682 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:28.902800 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:28.904641 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.905336 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.906550 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:28.907132 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:28.907285 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:28.911382 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:28.911389 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:28.911394 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:28.911436 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:28.914701 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.915410 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.915415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.915419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.915642 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:28.916040 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:28.916045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:28.916048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:28.916323 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:28.916613 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:28.916917 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:27:28.917162 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260807 18:27:28.917609 30550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: Bootstrap starting.
I20260807 18:27:28.922043 30550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:28.923148 30323 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:27:28.923395 30550 log.cc:824] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:28.926903 30550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: No bootstrap required, opened a new log
I20260807 18:27:28.925832 30268 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b92d718307eb426bbfd8aeaa1f964498"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60642
I20260807 18:27:28.939493 30550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:28.940018 30550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df4874115cc4447a8dd0d5ba029dc44, State: Initialized, Role: FOLLOWER
I20260807 18:27:28.940621 30550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:28.941748 30560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:28.942176 30560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:28.943032 30550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:27:28.953850 30576 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:27:28.954051 30576 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:27:28.951691 30456 master_runner.cc:186] Error getting master registration for 127.29.156.254:44409: OK, instance_id { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" instance_seqno: 1786127247684585 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:27:28.964334 30456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:27:29.001135 30439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 5
Aug 07 18:27:29 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127248, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.253@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:27:29.019815 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.002075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.29.156.252:33471)
0807 18:27:29.002413 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.002420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.002446 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:27:29.002652 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.002663 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.002761 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.002881 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.004066 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.005276 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.007289 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.008040 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.008196 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.012178 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:29.012186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:29.012192 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:29.012250 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:29.016362 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.017234 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.017240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.017244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.017412 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.017887 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.017892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.017896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.018421 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:29.018600 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:29.019086 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:27:29.019373 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 18:27:29.020675 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.001986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33064)
0807 18:27:29.002323 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:29.002350 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:29.002493 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:29.002936 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:29.002941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:29.003708 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:29.003899 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:29.003909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.005450 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.005456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:29.007100 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:29.007120 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.008217 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.008224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:29.008485 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.009238 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:29.009259 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:29.011921 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:29.016511 (+ 4590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:29.016517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:29.016521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:29.017078 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:29.017544 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:29.017549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:29.017553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:29.017768 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:29.018559 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:29.018565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:29.018739 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 18:27:29.018917 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 18:27:29.019285 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260807 18:27:29.027102 30439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: Bootstrap starting.
I20260807 18:27:29.032059 30439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:29.033448 30439 log.cc:824] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:29.036896 30439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: No bootstrap required, opened a new log
I20260807 18:27:29.048266 30439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.048705 30439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ea1e59080344928799582cbd50972b, State: Initialized, Role: FOLLOWER
I20260807 18:27:29.049275 30439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.050339 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.050783 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.051707 30439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:27:29.064185 30596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:27:29.064375 30596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:27:29.061096 30338 master_runner.cc:186] Error getting master registration for 127.29.156.254:44409: OK, instance_id { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" instance_seqno: 1786127247684585 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:27:29.072201 30338 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 18:27:29.074800 30489 tablet.cc:2369] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:27:29.112541 30308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } attempt: 5
Aug 07 18:27:29 dist-test-slave-sb2z krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.156.254@KRBTEST.COM for kudu/127.29.156.252@KRBTEST.COM
I20260807 18:27:29.132355 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.113401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.156.252:33471 (local address 127.0.0.1:33070)
0807 18:27:29.113668 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 18:27:29.113695 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:27:29.113832 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:27:29.114424 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:27:29.114430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:27:29.114974 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:27:29.115218 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:29.115230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.116935 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.116942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:29.118396 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:27:29.118412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.119805 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.119812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:27:29.120039 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.120852 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:27:29.120906 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:27:29.124004 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:27:29.129316 (+ 5312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:29.129322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:29.129326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:29.129884 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:29.130363 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:27:29.130368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:27:29.130371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:27:29.130555 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:27:29.131243 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:27:29.131248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:27:29.131430 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 18:27:29.131619 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:27:29.131916 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:27:29.132665 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.113853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.29.156.252:33471)
0807 18:27:29.114047 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.114054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.114078 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:27:29.114151 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.114157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.114250 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.114377 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.115438 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.116791 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.118578 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.119348 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.119513 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.124325 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:29.124334 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:29.124339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:29.124488 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:29.129167 (+ 4679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.130033 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.130040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.130045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.130240 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.130691 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.130697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.130701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.131121 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:29.131298 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:29.131738 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 18:27:29.132024 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:27:29.139735 30308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: Bootstrap starting.
I20260807 18:27:29.144974 30308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: Neither blocks nor log segments found. Creating new log.
I20260807 18:27:29.146840 30308 log.cc:824] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: Log is configured to *not* fsync() on all Append() calls
I20260807 18:27:29.150944 30308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: No bootstrap required, opened a new log
I20260807 18:27:29.160368 30308 raft_consensus.cc:348] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.160974 30308 raft_consensus.cc:729] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92d718307eb426bbfd8aeaa1f964498, State: Initialized, Role: FOLLOWER
I20260807 18:27:29.161628 30308 consensus_queue.cc:260] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.163102 30601 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.164295 30601 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.164525 30308 sys_catalog.cc:564] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:27:29.164604 30268 raft_consensus.cc:482] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:27:29.164829 30268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:29.168632 30268 raft_consensus.cc:504] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.171514 30408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b92d718307eb426bbfd8aeaa1f964498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "63ea1e59080344928799582cbd50972b"
I20260807 18:27:29.172200 30408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:27:29.173516 30242 tablet.cc:2369] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:27:29.176730 30408 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b92d718307eb426bbfd8aeaa1f964498 in term 1.
I20260807 18:27:29.177609 30235 leader_election.cc:302] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63ea1e59080344928799582cbd50972b, b92d718307eb426bbfd8aeaa1f964498; no voters:
I20260807 18:27:29.178176 30601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:27:29.182235 30519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b92d718307eb426bbfd8aeaa1f964498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7df4874115cc4447a8dd0d5ba029dc44"
I20260807 18:27:29.182813 30519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:27:29.186836 30519 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b92d718307eb426bbfd8aeaa1f964498 in term 1.
I20260807 18:27:29.188024 30268 leader_election.cc:288] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [CANDIDATE]: Term 1 election: Requested vote from peers 63ea1e59080344928799582cbd50972b (127.29.156.253:42959), 7df4874115cc4447a8dd0d5ba029dc44 (127.29.156.252:33471)
I20260807 18:27:29.196223 30601 raft_consensus.cc:686] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [term 1 LEADER]: Becoming Leader. State: Replica: b92d718307eb426bbfd8aeaa1f964498, State: Running, Role: LEADER
I20260807 18:27:29.197028 30601 consensus_queue.cc:237] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } }
I20260807 18:27:29.203874 30603 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92d718307eb426bbfd8aeaa1f964498. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.204507 30603 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:29.211732 30606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:27:29.228116 30606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:27:29.254320 30519 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [term 1 FOLLOWER]: Refusing update from remote peer b92d718307eb426bbfd8aeaa1f964498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:27:29.255678 30603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:27:29.257683 30377 tablet.cc:2369] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:27:29.263732 30408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [term 1 FOLLOWER]: Refusing update from remote peer b92d718307eb426bbfd8aeaa1f964498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:27:29.264926 30603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:27:29.302335 30560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92d718307eb426bbfd8aeaa1f964498. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.302696 30560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.308293 30645 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:27:29.309536 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.234666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.29.156.252:33471)
0807 18:27:29.234923 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.234932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.248743 (+ 13811us) server_negotiation.cc:407] Connection header received
0807 18:27:29.253912 (+ 5169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.253921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.254040 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.254128 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.270195 (+ 16067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.271617 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.281258 (+ 9641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.282185 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.282315 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.291263 (+ 8948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:29.291275 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:29.291283 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:29.291349 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:29.296132 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.304074 (+ 7942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.304083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.304089 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.304302 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.305718 (+ 1416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.305727 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.305732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.306233 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:29.306634 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:29.308854 (+ 2220us) server_negotiation.cc:299] Negotiation successful
0807 18:27:29.309174 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:27:29.312676 30624 mvcc.cc:205] Tried to move back new op lower bound from 7315977212913496064 to 7315977212718370816. Current Snapshot: MvccSnapshot[applied={T|T < 7315977212913496064}]
I20260807 18:27:29.316955 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.230199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.29.156.254:44409)
0807 18:27:29.230416 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.230426 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.248135 (+ 17709us) server_negotiation.cc:407] Connection header received
0807 18:27:29.250622 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.250632 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.250744 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.250866 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.266028 (+ 15162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.267590 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.279715 (+ 12125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.280715 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.280857 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.297454 (+ 16597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:29.297465 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:29.297473 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:29.297533 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:29.302143 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.304913 (+ 2770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.304923 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.304929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.305156 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.310021 (+ 4865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.310026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.310029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.311265 (+ 1236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:29.311567 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:29.312186 (+ 619us) server_negotiation.cc:299] Negotiation successful
0807 18:27:29.312447 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:27:29.325037 30637 mvcc.cc:205] Tried to move back new op lower bound from 7315977212913496064 to 7315977212718370816. Current Snapshot: MvccSnapshot[applied={T|T < 7315977212913496064}]
I20260807 18:27:29.330288 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:29.249372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.29.156.253:42959)
0807 18:27:29.249603 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.249624 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.260833 (+ 11209us) server_negotiation.cc:407] Connection header received
0807 18:27:29.262924 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.262934 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.263051 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.263144 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.274164 (+ 11020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.281652 (+ 7488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.295669 (+ 14017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:29.296553 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:29.296711 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:29.311773 (+ 15062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:27:29.311782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:27:29.311790 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:27:29.311845 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:27:29.316301 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.326911 (+ 10610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.326917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.326921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.327114 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:27:29.327693 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:27:29.327698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:27:29.327701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:27:29.328112 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:27:29.328358 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:29.329683 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0807 18:27:29.329940 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:27:29.337303 30560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.337607 30560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.340243 30606 catalog_manager.cc:1347] Generated new cluster ID: c4fc53aa1841464f9040e10ee8187b1d
I20260807 18:27:29.340364 30606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:27:29.340327 30601 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.340657 30601 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:29.342716 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92d718307eb426bbfd8aeaa1f964498. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.343053 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.361581 30603 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.361882 30603 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498 [sys.catalog]: This master's current role is: LEADER
I20260807 18:27:29.368116 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b92d718307eb426bbfd8aeaa1f964498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d718307eb426bbfd8aeaa1f964498" member_type: VOTER last_known_addr { host: "127.29.156.254" port: 44409 } } peers { permanent_uuid: "63ea1e59080344928799582cbd50972b" member_type: VOTER last_known_addr { host: "127.29.156.253" port: 42959 } } peers { permanent_uuid: "7df4874115cc4447a8dd0d5ba029dc44" member_type: VOTER last_known_addr { host: "127.29.156.252" port: 33471 } } }
I20260807 18:27:29.368407 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:27:29.393294 30606 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:27:29.399206 30606 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:27:29.421252 30606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b92d718307eb426bbfd8aeaa1f964498: Generated new TSK 0
I20260807 18:27:29.422484 30606 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:27:29.588387 30558 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.565129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.29.156.254:44409)
0807 18:27:29.565375 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.565383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.585996 (+ 20613us) server_negotiation.cc:407] Connection header received
0807 18:27:29.586625 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.586631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.586738 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.586823 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.588100 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60656: BlockingRecv error: recv got EOF from 127.0.0.1:60656 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:27:29.590916 30555 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.571256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.29.156.252:33471)
0807 18:27:29.571453 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.571462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.589209 (+ 17747us) server_negotiation.cc:407] Connection header received
0807 18:27:29.589448 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.589454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.589535 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.589603 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.590678 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33090: BlockingRecv error: recv got EOF from 127.0.0.1:33090 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:27:29.594801 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.592319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43130 (local address 127.29.156.253:42959)
0807 18:27:29.592556 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.592562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.593200 (+ 638us) server_negotiation.cc:407] Connection header received
0807 18:27:29.593435 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.593441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.593549 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.593628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.594552 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43130: BlockingRecv error: recv got EOF from 127.0.0.1:43130 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:27:29.630395 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.628044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.29.156.253:42959)
0807 18:27:29.628255 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.628262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.628730 (+ 468us) server_negotiation.cc:407] Connection header received
0807 18:27:29.629023 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.629027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.629133 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.629265 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.630083 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43144: BlockingRecv error: recv got EOF from 127.0.0.1:43144 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:27:29.638104 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.636037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.29.156.253:42959)
0807 18:27:29.636226 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.636235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.636619 (+ 384us) server_negotiation.cc:407] Connection header received
0807 18:27:29.636846 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.636852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.636937 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.637074 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.637872 (+ 798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43158: BlockingRecv error: recv got EOF from 127.0.0.1:43158 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 18:27:29.656077 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.653680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.29.156.253:42959)
0807 18:27:29.653925 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.653935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.654502 (+ 567us) server_negotiation.cc:407] Connection header received
0807 18:27:29.654740 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.654745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.654849 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.654968 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.655800 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43174: BlockingRecv error: recv got EOF from 127.0.0.1:43174 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 18:27:29.665484 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.663389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.29.156.253:42959)
0807 18:27:29.663665 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.663671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.664014 (+ 343us) server_negotiation.cc:407] Connection header received
0807 18:27:29.664256 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.664261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.664361 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.664483 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.665231 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43182: BlockingRecv error: recv got EOF from 127.0.0.1:43182 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260807 18:27:29.679574 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.677165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.29.156.253:42959)
0807 18:27:29.677387 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.677396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.677998 (+ 602us) server_negotiation.cc:407] Connection header received
0807 18:27:29.678234 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.678245 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.678324 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.678455 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.679279 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43194: BlockingRecv error: recv got EOF from 127.0.0.1:43194 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:27:29.716797 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.714348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.29.156.253:42959)
0807 18:27:29.714564 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.714571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.715220 (+ 649us) server_negotiation.cc:407] Connection header received
0807 18:27:29.715426 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.715431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.715543 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.715650 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.716567 (+ 917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43204: BlockingRecv error: recv got EOF from 127.0.0.1:43204 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:27:29.755505 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.753037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.29.156.253:42959)
0807 18:27:29.753347 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.753354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.753751 (+ 397us) server_negotiation.cc:407] Connection header received
0807 18:27:29.753977 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.753982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.754069 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.754184 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.755204 (+ 1020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260807 18:27:29.825534 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.823296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.29.156.253:42959)
0807 18:27:29.823605 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.823612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.823993 (+ 381us) server_negotiation.cc:407] Connection header received
0807 18:27:29.824173 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.824181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.824310 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.824450 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.825292 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43210: BlockingRecv error: recv got EOF from 127.0.0.1:43210 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:27:29.956357 30576 catalog_manager.cc:1259] Loaded cluster ID: c4fc53aa1841464f9040e10ee8187b1d
I20260807 18:27:29.956462 30576 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: loading cluster ID for follower catalog manager: success
W20260807 18:27:29.959801 30447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:27:29.957699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.29.156.253:42959)
0807 18:27:29.957925 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:29.957931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:29.958248 (+ 317us) server_negotiation.cc:407] Connection header received
0807 18:27:29.958468 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:29.958472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:29.958546 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:29.958645 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:27:29.959511 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43222: BlockingRecv error: recv got EOF from 127.0.0.1:43222 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:27:29.960026 30576 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: acquiring CA information for follower catalog manager: success
I20260807 18:27:29.962218 30576 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7df4874115cc4447a8dd0d5ba029dc44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:27:30.066591 30596 catalog_manager.cc:1259] Loaded cluster ID: c4fc53aa1841464f9040e10ee8187b1d
I20260807 18:27:30.066700 30596 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: loading cluster ID for follower catalog manager: success
I20260807 18:27:30.069319 30596 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: acquiring CA information for follower catalog manager: success
I20260807 18:27:30.071069 30596 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63ea1e59080344928799582cbd50972b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:27:30.227762 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:30.221612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.29.156.253:42959)
0807 18:27:30.221895 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:30.221903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:30.222143 (+ 240us) server_negotiation.cc:407] Connection header received
0807 18:27:30.222424 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:30.222429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:30.222596 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:30.222731 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:30.223222 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.223961 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.225442 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.225838 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.225981 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:30.226504 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:30.227066 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:30.227273 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:30.227341 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 18:27:30.227439 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 18:27:30.240314 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:30.234244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.29.156.252:33471)
0807 18:27:30.234772 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:30.234778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:30.234799 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:27:30.234922 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:30.234933 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:30.235110 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:30.235293 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:30.235783 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.236554 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.237918 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.238337 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.238490 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:30.239013 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:30.239649 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:30.239811 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:30.239869 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:27:30.239955 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":265,"threads_started":1}
I20260807 18:27:30.256999 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:30.247368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.29.156.254:44409)
0807 18:27:30.249789 (+ 2421us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:30.249796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:30.249819 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:27:30.249961 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:30.249967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:30.250214 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:30.250403 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:30.250828 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.251921 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.254308 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.254718 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.254855 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:30.255489 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:30.256014 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:30.256093 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:30.256243 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:27:30.256316 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2283,"thread_start_us":2126,"threads_started":1}
I20260807 18:27:30.257596 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:30.248405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.29.156.253:42959)
0807 18:27:30.248641 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:30.248650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:30.250181 (+ 1531us) server_negotiation.cc:407] Connection header received
0807 18:27:30.250424 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:30.250430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:30.250517 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:30.250593 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:30.251497 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.252616 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.255148 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.255776 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.255906 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:30.256873 (+ 967us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:30.257006 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:30.257108 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:30.257226 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 18:27:30.257305 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:27:30.258172 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:27:30.247565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.29.156.252:33471)
0807 18:27:30.247808 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:27:30.247815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:27:30.249726 (+ 1911us) server_negotiation.cc:407] Connection header received
0807 18:27:30.250068 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:27:30.250074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:27:30.250189 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:27:30.250280 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:27:30.251265 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.252443 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.255065 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:27:30.255695 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:27:30.255819 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:27:30.256810 (+ 991us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:27:30.256922 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:27:30.257027 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:27:30.257828 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 18:27:30.257916 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:27:30.263140 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30210
I20260807 18:27:30.264276 30301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:30.377164 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30210
I20260807 18:27:30.408095 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30338
I20260807 18:27:30.409305 30430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:30.521317 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30338
I20260807 18:27:30.550303 30323 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskyv84zy/build/asan/bin/kudu pid 30456
I20260807 18:27:30.551767 30543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:27:30.662556 30323 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv84zy/build/asan/bin/kudu with pid 30456
2026-08-07T18:27:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5200 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10642 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (402042 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST